← 返回列表

SonarQube实现自动化代码扫描

单站 bloodzer0 发布 Mon, 27 Apr 2020 20:35:00 +0800 入库 2026-11-18T06:46:06

原创 bloodzer0 2020-04-27 20:35

记录使用SonarQube实现DevSecOps中代码扫描的经历

# 解决依赖,SonarQube7.9必须是java11才可以运行``yum install java-11-openjdk.x86_64 java-11-openjdk-devel.x86_64 -y

# 安装PG数据库,新版本已经不支持MySQL了``wget https://download.postgresql.org/pub/repos/yum/reporpms/EL-7-x86_64/pgdg-redhat-repo-latest.noarch.rpmrpm -ivh pgdg-redhat-repo-latest.noarch.rpm``yum install postgresql10.x86_64 postgresql10-server.x86_64 postgresql10-libs.x86_64 postgresql10-libs.x86_64 postgresql-jdbc.noarch -y

# 初始化PG``/usr/pgsql-10/bin/postgresql-10-setup initdb

# 允许本机连接``vim /var/lib/pgsql/10/data/pg_hba.conf

# 启动PG``systemctl enable postgresql-10.service``systemctl start postgresql-10.service

# 配置PG``su - postgres``psql

# 将template1取消设置为模板``UPDATE pg_database SET datistemplate = FALSE WHERE datname = 'template1';

# 删除template1``DROP DATABASE template1;

# 重新创建template1,并设置默认编码``CREATE DATABASE template1 WITH TEMPLATE = template0 ENCODING = 'UTF8';

# 将template1重新设置为模板``UPDATE pg_database SET datistemplate = TRUE WHERE datname = 'template1';

# 创建用户``create user sonarqube with password 'sonarqube';``alter role sonarqube login;

# 创建数据库``create database sonarqube encoding='UTF8' owner sonarqube;

# 创建用户``useradd sonarqube``passwd sonarqube``su - sonarqube

# 安装SonarQube``wget https://binaries.sonarsource.com/Distribution/sonarqube/sonarqube-8.0.zipunzip sonarqube-8.0.zip && cd sonarqube-8.0

# 修改配置文件``vim /home/sonarqube/sonarqube-8.0/conf/sonar.properties

sonar.jdbc.username=sonarqube``sonar.jdbc.password=sonarqube``sonar.jdbc.url=jdbc:postgresql://127.0.0.1/sonarqube

# 配置Java版本``# 由于8.0需要Java11,但是安装PG10的时候系统会默认安装Java1.8,所以需要配置Java地址``vim /home/sonarqube/sonarqube-8.0/conf/wrapper.conf

# 我是yum安装的java11,如果大家是配置或者编译的需要使用对应的java地址``wrapper.java.command=/usr/lib/jvm/java-11-openjdk-11.0.6.10-1.el7_7.x86_64/bin/java

# 启动服务``./bin/linux-x86-64/sonar.sh start``访问:http://10.10.10.12:9000,账号密码:admin/admin。

[1]: max file descriptors [4096] for elasticsearch process is too low, increase to at least [65535]``[2]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]

vim /etc/sysctl.conf
# 添加如下内容``vm.max_map_count=655360

# 立即生效``sysctl -p

vim /etc/security/limits.conf

# 添加如下内容``* soft nofile 65536``* hard nofile 65536

# limits不需要重启,但是需要重新登录

# 下载SonarScanner``wget https://binaries.sonarsource.com/Distribution/sonar-scanner-cli/sonar-scanner-cli-4.2.0.1873-linux.zip

unzip sonar-scanner-cli-4.2.0.1873-linux.zip -d /home/sonarqube/

  1. 在SonarQube管理⻚面创建一个Token:68bec86cee8ec9a9ad8621592670c9939e6ba71e
  2. 执行扫描

/home/sonarqube/sonar-scanner-4.2.0.1873-linux/bin/sonar-scanner -Dsonar.host.url=http://127.0.0.1:9000 -Dsonar.login=68bec86cee8ec9a9ad8621592670c9939e6ba71e -Dsonar.projectKey=java-sec-code_scanner -Dsonar.java.binaries=/home/sonarqube/projects/java-sec-code
- 在应用市场中下载安装
- 在GitHub下载jar文件放置到SonarQube的插件目录中

mvn clean package -DskipTests

/home/sonarqube/sonar-scanner-4.2.0.1873-linux/bin/sonar-scanner -Dsonar.host.url=http://127.0.0.1:9000 -Dsonar.login=68bec86cee8ec9a9ad8621592670c9939e6ba71e -Dsonar.projectKey=java-sec-code_findbugs -Dsonar.java.binaries=/home/sonarqube/projects/java-sec-code

# 不更新漏洞库执行扫描``/home/sonarqube/dependency-check/bin/dependency-check.sh -n

# 只更新漏洞库库,不执行扫描``/home/sonarqube/dependency-check/bin/dependency-check.sh --updateonly

# 使用代理更新漏洞库``/home/sonarqube/dependency-check/bin/dependency-check.sh --updateonly --proxyserver host --proxyport port

# 编译项目``mvn clean package -DskipTests

# 生成xml报告``/home/sonarqube/dependency-check/bin/dependency-check.sh -s /home/sonarqube/projects/java-sec-code/target/java-sec-code-1.0.0.jar -f XML -o /home/sonarqube/reports/java-sec-code-report.xml

# 在5.3.2 版本中,小B遇到了一个坑就是第一项执行DependencyCheck时,当执行到 Central Analyzer 分析时非常缓慢(由于网络原因导致的),可以使用--disableCentral禁用,但是官方文档中不推荐禁用,所以只能在执行是使用代理或调整网络来解决

# 生成html报告``/home/sonarqube/dependency-check/bin/dependency-check.sh -s /home/sonarqube/projects/java-sec-code/target/java-sec-code-1.0.0.jar -f HTML -o /home/sonarqube/reports/java-sec-code-report.html

# 将报告导入SonarQube并执行SonarScanner扫描,这里我们使用的是FindBugs Security Audit审计规则``/home/sonarqube/sonar-scanner-4.2.0.1873-linux/bin/sonar-scanner -Dsonar.host.url=http://127.0.0.1:9000 -Dsonar.login=68bec86cee8ec9a9ad8621592670c9939e6ba71e -Dsonar.projectKey=java-sec-code_dependency -Dsonar.java.binaries=/home/sonarqube/projects/java-sec-code -Dsonar.dependencyCheck.reportPath=/home/sonarqube/reports/java-sec-code-report.xml -Dsonar.dependencyCheck.htmlReportPath=/home/sonarqube/reports/java-sec-code-report.html

stages: - sast

sast: stage:sast script: - mvn clean package -DskipTests # 编译项目 - mkdir $PWD/reports # 创建报告目录 - /home/gitlab-runner/dependency-check/bin/dependency-check.sh -s $PWD/ -f XML -o $PWD/reports/dependency-check-report.xml # 执行扫描,并生成xml报告 - /home/gitlab-runner/dependency-check/bin/dependency-check.sh -s $PWD/ -f HTML -o $PWD/reports/dependency-check-report.html # 执行扫描,并生成html报告 - sonar-scanner -Dsonar.host.url=http://10.10.10.12:9000 -Dsonar.login=token -Dsonar.projectKey=$CI_PROJECT_NAME:$CI_COMMIT_REF_NAME -Dsonar.java.binaries=$PWD -Dsonar.dependencyCheck.reportPath=$PWD/reports/dependency-check-report.xml -Dsonar.dependencyCheck.htmlReportPath=$PWD/reports/dependency-check-report.html # 执行扫描,并将报告传输给SonarQube tags: - security

import requests

def get_vul(project_name): url = "http://172.16.26.13:9000/api/issues/search?project=%s&types=VULNERABILITY" % project_name vulnerability_number = requests.get(url,auth=('admin','admin')).json()['paging']['total']

`for row in requests.get(url,auth=('admin','admin')).json()['issues']:`        `if row['status'] == "OPEN":`            `if row['severity'] == "MAJOR" or row['severity'] == "CRITICAL" or row['severity'] == "BLOCKER":`                `alarm_data = ""`            `else:`                `pass`        `else:`            `pass`

`print("漏洞数量: ", vulnerability_number)`

if __name__ == '__main__': project_name = "java-sec-code_dependency" get_vul(project_name)

sonar.jdbc.url=jdbc:mysql://address:3306/sonar?useUnicode=true&characterEncoding=utf8
ERROR: Unable to load component class org.sonar.scanner.report.MetadataPublisher``ERROR: Caused by: Unable to load component class org.sonar.scanner.rule.QualityProfiles

用CI_COMMIT_REF_NAME,但是如果使用这个变量在项目名称中包含/、#等特殊字符时,Gitlab-CI的pipeline会执行失败。这个时候我们选择CI_COMMIT_REF_SLUG(先将$CI_COMMIT_REF_NAME的值转换成小写,最大不能超过63个字节,然后把除了0-9和a-z的其他字符转换成-。在URLs和域名名称中使用。)

原文链接:https://mp.weixin.qq.com/s?__biz=MzU2NzY5MDY3MQ==&mid=2247483990&idx=1&sn=23ed7ebac98f73f98242a43c0933660b