Advanced outlier detection settings

Tags: Flow

Overview

Currently our default outlier detection is set-up to mark any commit over one thousand lines of code as an outlier. With Advanced outlier detection you can tailor your outlier commits to your organizational needs. Below will instruct you how to use these settings. Also see our help doc on how to manually mark a commit as an outlier. 

Getting started

In order to adjust the outlier detection settings you first need to have the permissions to do so.

Step 1: Go to the top navigation bar and click Settings. Using the left navigation under User Management, click Users.


Step 2: Locate the User you want to give access to the Advanced outlier detection settings and open their User detail page. Under Access tab click on Administration.


Step 3: Give them access to both manage configurations and manage outlier commits and then click Save.

  • Manage configurations - This permission gives the User access Advanced Outlier Detection Settings plus any general report configurations.
  • Manage commits - This permission gives the User access to manually mark commits as outliers in the Work log report.



Step 4: After clicking Save Changes, a pop-up will appear asking to confirm your changes. Click Yes, update user's view access.


Step 4: You should now see Configurations in your Settings navigation menu. Under Report Settings click Configurations.


Step 5: On the Configurations page, there are three different ways you can customize your outlier detection. You can use one or a combination of the three to set up your outlier rules. 

A). Impact - dictate which commits are marked as an outlier based on the raw Impact score or by a percentage range of all impact scores of commits that we’ve processed for your organization (0-25%). 

B). Size - dictate which commits are marked as an outlier based on the size (number of LoC) of the commit. Here you can also ignore whitespace, comments or lines with a certain number of characters.  

C). Regular Expressions - dictate which commits are marked as an outlier using regular expressions. 


A). Impact

You can determine which commits are outliers by setting filters for Impact scores.

1. Ignore commits by Impact percentage. Here you can input a percentage based on Impact score to determine which commits to include as an outlier. In the example below any commit that falls into the highest 20% Impact score or the lowest 3% Impact score of your processed commits will be considered an outlier. 

Tip: To get an idea for what raw Impact values are for your team, review the Fundamentals Impact report, and base your outlier Impact score off of your team’s data. You must use a percentage greater than 0.

2. Ignore commits by raw Impact score. In this setting you can ignore commits by Impact score. In this example any commit that has an impact score above 500 or below 10 will be considered an outlier. 


B). Size

You can determine which commits are outliers by the size of the commit. In this setting you can set the following:

1. The minimum number of lines needed in order to be considered an outlier. In the example below any commit that has less than 3 lines of code will be marked as an outlier. 


2. You can determine what lines to count when considering a commit for an outlier. You can:

  • Ignore whitespace - this will eliminate any lines that do not contain any characters

  • Ignore comments - this will eliminate any lines that contain comments

  • Ignore lines that contain less than a designated number of non-whitespace characters. This will include lines without any characters. 


3. The maximum number of lines needed in order to be considered an outlier. In the example below, any commit that has 1000 or more lines of code will be marked as an outlier. 


C). Regular expression

You can determine which commits are outliers by using regular expressions to match on the User associated with the commit or on the commit message. You can create as many rules here as you need. 

Step 1: Click New Rule.


Step 2: Type in your regular expression and choose whether to "match on user" and "match on message" or just one.


Testing regular expressions

If you have commits that you know should be marked as an outlier and should be included within your regular expression your can test you regular expression to determine which commits it will include. Click Test a Regular Expression.


Step 1: A modal will appear where you need to type in the following:

  1. Your regular expression.
  2. Commit SHA(s) to test against specific commits- you can put in the entire SHA number or just the first 6 characters.
  3. Indicate whether you want to “Match on user” or “Match on message”.

Step 2: Click Load commits and show matches. You will now see the 3 commits listed below and if they match your regular expression.

  • Matches - will be considered outliers based on your regular expression

  • Does Not Match - will remain a regular commit within your reports


Step 3: If you do not have commits you want to test your regular expression with, leave Commits to test blank to load your 100 most recent commits.


Finishing up

When you are done making changes to your outlier detection settings, you need to make sure to save your changes. 

Step 1: Click Save Changes.


Step 2: You will need to confirm again to save changes. Click Yes, save changes.


Step 3: You will then be asked whether you want to score all past commits using the new settings or only new commits. A few things to keep in mind when re-scoring past commits:

  • Scoring all past commits may cause a delay in processing new commit data. Depending on the size of your organization it can take anywhere from 10-15 minutes to a few hours to re-score your past commits. You can always re-score past commits at a later time by clicking the Re-score all past commits link at the top of your Advanced outlier detection settings page

  • If you choose to postpone your re-score, you will still see outlier commits if they are larger than 1k LoC (default outlier setting) and any new code will be scored based on your new outlier settings. 


Step 4: You successfully added and saved your outlier detection settings. If you have re-scored your past commit data, you will now be able to see your outlier commits in your Work log report. Any outlier commit will appear with a red line through it as illustrated below. 



What happens when I turn off my Advanced outlier detection settings? 

If you turn off your Advanced outlier detection settings, you do not need to do anything. Your organization will return to the default outlier settings of any commit larger than 1000 lines will be considered an outlier.

If you turn off your outlier settings, it will also return any manually marked commit to its original state.

Advanced outlier detection use cases

Below are a few use cases, and the necessary settings to implement them in your own account. 

Default outlier detection (commits larger than 1000 lines) AND manually marked commits

If you would like to keep the current default outlier plus be able to manually mark commits as outliers, use the following settings: 


No automatic filters BUT manually marked commits 

If you would like to include all of your commits except the ones you manually mark as outliers, use the following settings. 



back to top


If you need help, please email support@pluralsight.com for 24/7 assistance.