Opsera now offers Git Scraper support in a pipeline step. This allows user to choose from scraper libraries then run a scan against the configured SCM repos. Define a maximum threshold and pipeline will use the values to determine the final status of the step. Use the following documentation as a guideline of how to use Opsera’s Git Scraper support in pipelines.

Git Scraper Pipeline Configuration

1. Create a new pipeline step. In Step Setup, choose Gitscraper from the Tool drop-down.

   

2. Click the ‘Save’ button and close the step.

3. Click the gear icon (⚙︎) to configure the step.

   

4. In the Step Configuration, select the following values from the provided drop-downs:
   

   

   
   Scraper Module - Select a scraper module from the drop-down. Choose ‘Trufflehog’ or ‘Gitleaks’
   Source Code Management Tool Type - Select a Source Code Management Tool Type from the drop-down. Choose Bitbucket, Github or GitLab.
   Source Code Management Tool - Select the respective Source Code Management Tool containing the repository to run a scan against.
   Repository - Select the repository to run a scan against.
   Branch - Select the branch to run a scan against.
   Maximum Allows Secrets Threshold - Select the maximum allowed secrets that scan can contain without receiving a failure status.
   Exclude Certain Secrets from Scan - Secrets to Ignore: Select any Parameters to be ignored during the scan from the drop-down. These will not be counted toward the threshold. To add them to the table, click the ‘Add’ button. Parameters are fetched from Tool Registry Parameters.

   

   
   Exclude Certain Project Files from Scan - Project Files to Ignore: Select Absolute File Path(s) to be ignored during the scan from the drop-down. These will not be counted toward the threshold. To add them to the table, click the ‘Add’ button.

   

   
   

5. Click the ‘Save’ button to save configurations and trigger the pipeline.

Git Scraper Report in Pipeline Logs

Once the pipeline has run, view the git scraper report in Pipeline Logs to analyze the results.

Scenario 1: Scan completed successfully and results are within the Threshold limit

This scan had a provided threshold of 10 and 3 files were included in the results.

1. In Pipeline Logs, click the ‘Report’ row. In the following screenshot, the results exceeded the threshold number provided in the step configuration.

   

2. View the Git Scraper Execution Summary. The threshold provided for this scan was 10. There are 3 results appearing in the result below. Since the number of results did not meet the threshold, the step had a ‘Success’ status.

   

Scenario 2: Scan Failed or the results have exceeded the threshold

This scan had a provided threshold of 0 and 3 files were included in the results.

1. In Pipeline Logs, click the ‘Report’ row. In the following screenshot, the results exceeded the threshold number provided in the step configuration.

   

2. View the Git Scraper Execution Summary. The threshold provided for this scan was 0 There are 3 results appearing in the result below. Since the number of results did not meet the threshold, the step had a ‘Failure’ status.

   

Git Scraper KPI

View Git Scraper metrics from scans using the Git Scraper Metrics KPI in Insights. The metrics provide the number and trend result (compared to last scan) of Scanned Repositories, Clean Repositories, and Total Number of Issues. The Last Scan results are also included.

Scanned Repositories - The number of scanned repositories in the provided date range.

Clean Repositories - The number of clean repositories in the provided date range.

Total Number of Issues - The number of issues existing in the provided date range.