Using Context Rules

You can use the surrounding context of a match to help determine how likely it is that your potential match should actually be considered as a match by adjusting its confidence rating.

You can also tell the Detection Rule to return a portion of the surrounding context for manual review.

In the following example, in addition to providing a regular expression to match Social Security Numbers, we also look to see if someone has written the text “SSN” before and after the match, which might be a label indicating it is indeed a social security number. In which case, we change our confidence score to “VERY_LIKELY.” We then provide two possible matches in our payload, the first of which contains the string “SSN”.

curl --location --request POST 'https://api.nightfall.ai/v3/scan' \
--header 'Accept: application/json' \
--header 'Authorization: Bearer NF-rEpLaCeM3w1ThYoUrNiGhTfAlLKeY123' \
--header 'Content-Type: application/json' \
--data-raw '{
     "policy": {
          "detectionRules": [
               {
                    "detectors": [
                         {
                              "regex": {
                                   "isCaseSensitive": false,
                                   "pattern": "\\d{3}-\\d{2}-\\d{4}"
                              },
                              "contextRules": [
                                   {
                                        "regex": {
                                             "pattern": "SSN",
                                             "isCaseSensitive": false
                                        },
                                        "proximity": {
                                             "windowBefore": 20,
                                             "windowAfter": 20
                                        },
                                        "confidenceAdjustment": {
                                             "fixedConfidence": "VERY_LIKELY"
                                        }
                                   }
                              ],
                              "minNumFindings": 1,
                              "minConfidence": "POSSIBLE",
                              "detectorType": "REGEX",
                              "displayName": "SSN Match Detector"
                         }
                    ],
                    "name": "SSN Match Detection Rule",
                    "logicalOp": "ALL"
               }
          ],
          "contextBytes": 20
     },
     "payload": [
          "My SSN is 555-55-5555",
          "Here it is : 555-55-5555"
     ]
}
'

In the results, you can see the confidence for the first finding in the payload has been set to VERY_LIKELY while the second item is only LIKELY.

{
   "findings":[
      [
         {
            "finding":"555-55-5555",
            "beforeContext":"My SSN is ",
            "detector":{
               "name":"SSN Match Detector",
               "uuid":"6131f41c-dbdd-47a9-8c6f-1819c9baf388"
            },
            "confidence":"VERY_LIKELY",
            "location":{
               "byteRange":{
                  "start":10,
                  "end":21
               },
               "codepointRange":{
                  "start":10,
                  "end":21
               },
               "rowRange":null,
               "columnRange":null,
               "commitHash":""
            },
            "matchedDetectionRuleUUIDs":[
               
            ],
            "matchedDetectionRules":[
               "SSN Match Detection Rule"
            ]
         }
      ],
      [
         {
            "finding":"555-55-5555",
            "beforeContext":"Here it is : ",
            "detector":{
               "name":"SSN Match Detector",
               "uuid":"6131f41c-dbdd-47a9-8c6f-1819c9baf388"
            },
            "confidence":"LIKELY",
            "location":{
               "byteRange":{
                  "start":13,
                  "end":24
               },
               "codepointRange":{
                  "start":13,
                  "end":24
               },
               "rowRange":null,
               "columnRange":null,
               "commitHash":""
            },
            "matchedDetectionRuleUUIDs":[
               
            ],
            "matchedDetectionRules":[
               "SSN Match Detection Rule"
            ]
         }
      ]
   ],
   "redactedPayload":[
      "",
      ""
   ]
}