- 移除tools自动安装配置,改用环境检测 - 修复所有sh和cleanWs步骤的上下文问题 - 增加异常处理和错误恢复机制 - 支持Maven Wrapper作为fallback - 添加Jenkins工具配置和问题排查指南
321 lines
6.9 KiB
Markdown
321 lines
6.9 KiB
Markdown
# Jenkins流水线问题排查指南
|
||
|
||
## 🐛 常见问题及解决方案
|
||
|
||
### 1. JDK自动安装失败
|
||
|
||
#### 错误信息
|
||
```
|
||
Exception: hudson.tools.InstallationException: java.util.concurrent.ExecutionException: java.io.IOException: Failed to download from https://github.com/adoptium/temurin17-binaries/releases/download/...
|
||
```
|
||
|
||
#### 原因分析
|
||
- 网络连接问题,无法从GitHub下载JDK
|
||
- 防火墙或代理设置阻止下载
|
||
- GitHub下载服务器不稳定
|
||
|
||
#### 解决方案
|
||
1. **使用系统预安装JDK**(推荐)
|
||
```bash
|
||
# 在Jenkins节点上安装JDK
|
||
sudo apt update
|
||
sudo apt install openjdk-17-jdk
|
||
|
||
# 验证安装
|
||
java -version
|
||
which java
|
||
```
|
||
|
||
2. **配置Jenkins工具**
|
||
- 进入:`Manage Jenkins -> Global Tool Configuration`
|
||
- JDK部分:取消`Install automatically`
|
||
- 设置JAVA_HOME为实际路径
|
||
|
||
3. **修改流水线**
|
||
- 移除`tools`块中的JDK配置
|
||
- 使用环境检测机制
|
||
|
||
### 2. Maven自动安装失败
|
||
|
||
#### 错误信息
|
||
```
|
||
Exception downloading Maven: Connection timed out
|
||
```
|
||
|
||
#### 解决方案
|
||
1. **使用系统预安装Maven**
|
||
```bash
|
||
# 安装Maven
|
||
sudo apt install maven
|
||
|
||
# 或手动安装
|
||
cd /opt
|
||
sudo wget https://dlcdn.apache.org/maven/maven-3/3.9.6/binaries/apache-maven-3.9.6-bin.tar.gz
|
||
sudo tar xzf apache-maven-3.9.6-bin.tar.gz
|
||
sudo ln -s apache-maven-3.9.6 maven
|
||
```
|
||
|
||
2. **使用项目自带Maven Wrapper**
|
||
```bash
|
||
# 项目中已包含mvnw,可直接使用
|
||
./mvnw -version
|
||
```
|
||
|
||
### 3. MissingContextVariableException
|
||
|
||
#### 错误信息
|
||
```
|
||
WorkflowScript: 123: Invalid step "sh" used in "post" section without "steps" wrapper
|
||
hudson.remoting.ProxyException: org.jenkinsci.plugins.workflow.steps.MissingContextVariableException
|
||
```
|
||
|
||
#### 原因分析
|
||
- `sh`、`cleanWs`等步骤必须在`node`上下文中执行
|
||
- `post`块中的某些步骤缺少`script`包装
|
||
|
||
#### 解决方案
|
||
1. **将所有sh命令包装在script块中**
|
||
```groovy
|
||
post {
|
||
always {
|
||
script {
|
||
try {
|
||
sh 'docker system prune -f'
|
||
} catch (Exception e) {
|
||
echo "清理失败: ${e.getMessage()}"
|
||
}
|
||
}
|
||
}
|
||
}
|
||
```
|
||
|
||
2. **使用try-catch包装敏感操作**
|
||
```groovy
|
||
script {
|
||
try {
|
||
cleanWs()
|
||
} catch (Exception e) {
|
||
echo "工作空间清理失败: ${e.getMessage()}"
|
||
}
|
||
}
|
||
```
|
||
|
||
### 4. Docker相关问题
|
||
|
||
#### 问题:Docker命令权限不足
|
||
```
|
||
docker: permission denied while trying to connect to the Docker daemon socket
|
||
```
|
||
|
||
#### 解决方案
|
||
```bash
|
||
# 将Jenkins用户添加到docker组
|
||
sudo usermod -aG docker jenkins
|
||
|
||
# 重启Jenkins服务
|
||
sudo systemctl restart jenkins
|
||
|
||
# 验证权限
|
||
sudo -u jenkins docker ps
|
||
```
|
||
|
||
#### 问题:Docker镜像构建失败
|
||
```
|
||
docker: Error response from daemon: Cannot connect to the Docker daemon at unix:///var/run/docker.sock
|
||
```
|
||
|
||
#### 解决方案
|
||
```bash
|
||
# 检查Docker服务状态
|
||
sudo systemctl status docker
|
||
|
||
# 启动Docker服务
|
||
sudo systemctl start docker
|
||
sudo systemctl enable docker
|
||
```
|
||
|
||
### 5. SSH连接问题
|
||
|
||
#### 问题:SSH认证失败
|
||
```
|
||
sshpass: command not found
|
||
```
|
||
|
||
#### 解决方案
|
||
```bash
|
||
# 在Jenkins节点上安装sshpass
|
||
sudo apt update
|
||
sudo apt install sshpass
|
||
|
||
# 或在CentOS/RHEL上
|
||
sudo yum install sshpass
|
||
```
|
||
|
||
#### 问题:SSH主机密钥验证失败
|
||
```
|
||
Host key verification failed
|
||
```
|
||
|
||
#### 解决方案
|
||
```bash
|
||
# 添加-o StrictHostKeyChecking=no参数
|
||
sshpass -p "$SSH_PASS" ssh -o StrictHostKeyChecking=no user@host "command"
|
||
```
|
||
|
||
### 6. SonarQube连接问题
|
||
|
||
#### 问题:连接超时
|
||
```
|
||
SonarQube server [http://116.62.163.84:15010] can not be reached
|
||
```
|
||
|
||
#### 解决方案
|
||
1. **检查SonarQube服务状态**
|
||
```bash
|
||
# 在SonarQube服务器上
|
||
docker ps | grep sonar
|
||
curl -I http://localhost:15010
|
||
```
|
||
|
||
2. **网络连通性测试**
|
||
```bash
|
||
# 在Jenkins节点上测试
|
||
telnet 116.62.163.84 15010
|
||
curl -I http://116.62.163.84:15010
|
||
```
|
||
|
||
3. **临时跳过SonarQube扫描**
|
||
- 流水线中已包含try-catch,扫描失败不会中断构建
|
||
|
||
### 7. 应用健康检查失败
|
||
|
||
#### 问题:健康检查端点不可达
|
||
```
|
||
curl: (7) Failed to connect to 116.62.163.84 port 80: Connection refused
|
||
```
|
||
|
||
#### 解决方案
|
||
1. **检查应用端口映射**
|
||
```bash
|
||
# 检查容器运行状态
|
||
docker ps | grep jenkins-demo
|
||
|
||
# 检查端口监听
|
||
netstat -tlnp | grep :80
|
||
```
|
||
|
||
2. **检查防火墙设置**
|
||
```bash
|
||
# Ubuntu
|
||
sudo ufw status
|
||
sudo ufw allow 80
|
||
|
||
# CentOS
|
||
sudo firewall-cmd --permanent --add-port=80/tcp
|
||
sudo firewall-cmd --reload
|
||
```
|
||
|
||
## 🔧 快速修复脚本
|
||
|
||
### Jenkins节点环境准备脚本
|
||
```bash
|
||
#!/bin/bash
|
||
# jenkins-node-setup.sh
|
||
|
||
echo "🔧 准备Jenkins节点环境..."
|
||
|
||
# 安装基础软件
|
||
sudo apt update
|
||
sudo apt install -y openjdk-17-jdk maven git curl wget sshpass
|
||
|
||
# 安装Docker
|
||
if ! command -v docker &> /dev/null; then
|
||
curl -fsSL https://get.docker.com -o get-docker.sh
|
||
sudo sh get-docker.sh
|
||
sudo usermod -aG docker jenkins
|
||
fi
|
||
|
||
# 验证安装
|
||
echo "📋 验证环境..."
|
||
java -version
|
||
mvn -version
|
||
docker --version
|
||
git --version
|
||
|
||
echo "✅ Jenkins节点环境准备完成!"
|
||
echo "⚠️ 请重启Jenkins服务以应用Docker组权限变更"
|
||
```
|
||
|
||
### Docker清理脚本
|
||
```bash
|
||
#!/bin/bash
|
||
# docker-cleanup.sh
|
||
|
||
echo "🧹 清理Docker资源..."
|
||
|
||
# 停止所有容器
|
||
docker stop $(docker ps -aq) 2>/dev/null || true
|
||
|
||
# 删除所有容器
|
||
docker rm $(docker ps -aq) 2>/dev/null || true
|
||
|
||
# 清理镜像
|
||
docker image prune -f
|
||
docker system prune -f
|
||
|
||
echo "✅ Docker清理完成!"
|
||
```
|
||
|
||
## 📋 故障排查检查清单
|
||
|
||
### 环境检查
|
||
- [ ] Jenkins节点已安装JDK 17
|
||
- [ ] Jenkins节点已安装Maven 3.9+
|
||
- [ ] Jenkins节点已安装Docker
|
||
- [ ] Jenkins用户在docker组中
|
||
- [ ] 安装了sshpass工具
|
||
|
||
### 网络检查
|
||
- [ ] 可以访问Git仓库
|
||
- [ ] 可以访问SonarQube服务器
|
||
- [ ] 可以SSH连接目标服务器
|
||
- [ ] 可以访问Maven中央仓库
|
||
|
||
### 权限检查
|
||
- [ ] Jenkins用户可以执行docker命令
|
||
- [ ] SSH凭据配置正确
|
||
- [ ] 工作空间有读写权限
|
||
|
||
### 服务检查
|
||
- [ ] SonarQube服务正常运行
|
||
- [ ] 目标服务器Docker服务正常
|
||
- [ ] 应用端口未被占用
|
||
|
||
## 🚨 紧急恢复步骤
|
||
|
||
如果流水线完全无法运行:
|
||
|
||
1. **重置Jenkins工具配置**
|
||
- 删除所有自动安装的工具
|
||
- 重新配置为手动安装路径
|
||
|
||
2. **简化流水线**
|
||
- 临时注释掉SonarQube扫描
|
||
- 临时注释掉Docker部署
|
||
- 只保留基本的编译和测试
|
||
|
||
3. **手动验证每个步骤**
|
||
```bash
|
||
# 在Jenkins节点上手动执行
|
||
git clone http://116.62.163.84:15006/wangtianqi/java_demo.git
|
||
cd java_demo
|
||
./mvnw clean compile test package
|
||
docker build -t jenkins-demo .
|
||
```
|
||
|
||
4. **逐步恢复功能**
|
||
- 先确保基本流程通过
|
||
- 再逐个添加高级功能
|
||
|
||
按照以上指南排查,应该能解决大部分Jenkins流水线问题。
|