Nightfall Documentation
  • Data Detection and Response
  • Posture Management
  • Data Exfiltration Prevention
  • Data Encryption
  • Developer APIs
  • Data Classification and Discovery
  • Welcome to Developer APIs Documentation
  • Introduction to Developer APIs
    • Overview
    • Quickstart
    • Use Cases
    • Authentication and Security
  • Key Concepts
    • Entities and Terms to Know
    • Setting Up Nightfall
      • Creating API Key
      • Creating Detectors
      • Creating Detection Rules
      • Creating Policies
    • Alerting
    • Scanning Text
    • Scanning Files
      • Supported File Types
      • File Scanning and Webhooks
      • Uploading and Scanning API Calls
      • Special File Types
      • Specialized File Detectors
      • Webhooks and Asynchronous Notifications
        • Accessing Your Webhook Signing Key
        • Creating a Webhook Server
    • Scanning Features
      • Using Pre-Configured Detection Rules
        • Scanning Images for patterns using Custom Regex Detectors
      • Creating an Inline Detection Rule
      • Using Exclusion Rules
      • Using Context Rules
      • Using Redaction
      • Using Policies to Send Alerts
      • Detecting Secrets
      • PHI Detection Rules
    • Detector Glossary
    • Test Datasets
    • Errors
    • Nightfall Playground
  • Nightfall APIs
    • DLP APIs - Firewall for AI Platform
      • Rate Limits for Firewall APIs
    • DLP APIs - Native SaaS Apps
      • Policy User Scope Update API
      • Rate Limits for Native SaaS app APIs
  • Exfiltration Prevention APIs
    • Default
    • Models
  • Posture Management APIs
    • Default
    • Models
  • Nightfall Software Development Kit (SDK)
    • Overview
    • Java SDK
    • Python SDK
    • Go SDK
    • Node.JS SDK
  • Language Specific Guides
    • Overview
    • Python
    • Ruby
    • Java
  • Tutorials
    • GenAI Protection
      • OpenAI Prompt Sanitization Tutorial
      • Anthropic Prompt Sanitization Tutorial
      • LangChain Prompt Sanitization Tutorial
    • SaaS Protection
      • HubSpot DLP Tutorial
      • Zendesk DLP Tutorial
    • Observability Protection
      • Datadog DLP Tutorial
      • New Relic DLP Tutorial
    • Datastore Protection
      • Airtable DLP Tutorial
      • Amazon Kinesis DLP Tutorial
      • Amazon RDS DLP Tutorial
      • Amazon RDS DLP Tutorial - Full Scan
      • Amazon S3 DLP Tutorial
      • Elasticsearch DLP Tutorial
      • Snowflake DLP Tutorial
  • Nightfall Use Cases
    • Overview
    • GenAI Content Filtering-How to prevent exposure of sensitive data
    • Redacting Sensitive Data in 4 Lines of Code
    • Detecting Sensitive Data in SMS Automations
    • Building Endpoint DLP to Detect PII on Your Machine in Real-Time
    • Deploy a File Scanner for Sensitive Data in 40 Lines of Code
    • Using Scan API (with Python)
  • FAQs
    • What Can I do with the Firewall for AI
    • How quickly can I get started with Firewall for AI?
    • What types of data can I scan with API?
    • What types of detectors are supported out of the box?
    • Can I customize or bring my own detectors?
    • What is the pricing model?
    • How do I know my data is secure?
    • How do I get in touch with you?
    • Can I test out the detection and my own detection rules before writing any code?
    • How does Nightfall support custom data types?
    • How does Nightfall's Firewall for AI differs from other solutions?
  • Nightfall Playground
  • Login to Nightfall
  • Contact Us
Powered by GitBook
On this page
  • Built In Detectors
  • Regular Expression Example
  • Word List Example

Was this helpful?

Export as PDF
  1. Key Concepts
  2. Scanning Features

Creating an Inline Detection Rule

PreviousScanning Images for patterns using Custom Regex DetectorsNextUsing Exclusion Rules

Last updated 10 months ago

Was this helpful?

In addition to using , you may define Detection Rules within the body of your scan method by either supplying:

  • the identifier of one of Nightfall's

  • the UUID of an a Detector defined through the UI

  • a Regular Expression

  • a Word List.

Built In Detectors

Out of the box, Nightfall comes with an of native detectors.

In the example below two of Nightfall's native Detectors (detectorType = "NIGHTFALL_DETECTOR") are being used:

  • US_SOCIAL_SECURITY_NUMBER

  • CREDIT_CARD_NUMBER.

When defining a Detection Rule in line, you configure the minimum (minConfidence) and minimum number of times the match must be found (minNumFindings) for the rule to be triggered. .

In the payload body, you can see that we are submitting a list of three different strings to scan (payload). The first will trigger the U.S. Social Security Detector. The last will trigger the credit card Detector. The middle example will trigger neither.

For more information on the parameters related to redaction, see .

curl --request POST \
     --url https://api.nightfall.ai/v3/scan \
     --header 'Authorization: Bearer NF-rEpLaCeM3w1ThYoUrNiGhTfAlLKeY123' \
     --header 'Content-Type: application/json' \
     --data '{
       "policy": {
            "detectionRules": [
                 {
                      "detectors": [
                           {
                                "detectorType": "NIGHTFALL_DETECTOR",
                                "nightfallDetector": "US_SOCIAL_SECURITY_NUMBER",
                                "minNumFindings": 1,
                                "minConfidence": "LIKELY",
                                "displayName": "US Social Security Number"
                           },
                           {
                                "detectorType": "NIGHTFALL_DETECTOR",
                                "nightfallDetector": "CREDIT_CARD_NUMBER",
                                "minNumFindings": 1,
                                "minConfidence": "LIKELY",
                                "displayName": "Credit Card Number",
                                "redactionConfig": {
                                    "maskConfig": {
                                        "maskingChar": "πŸ‘€",
                                        "charsToIgnore": ["-"]
                                    }
                                }
                           }
                      ],
                      "name": "My Match Rule",
                      "logicalOp": "ANY"
                 }
            ]
       },
       "payload": [
            "The customer social security number is 458-02-6124",
            "No PII in this string",
            "My credit card number is 5310-2768-6832-9293"
       ]
     }'

Below is the response payload to the previous request.

{
  "findings": [
    [
      {
        "finding": "458-02-6124",
        "detector": {
          "name": "US Social Security Number",
          "uuid": "e30d9a87-f6c7-46b9-a8f4-16547901e069"
        },
        "confidence": "VERY_LIKELY",
        "location": {
          "byteRange": {
            "start": 39,
            "end": 50
          },
          "codepointRange": {
            "start": 39,
            "end": 50
          }
        },
        "matchedDetectionRuleUUIDs": [],
        "matchedDetectionRules": [
          "My Match Rule"
        ]
      }
    ],
    [],
    [
      {
        "finding": "5310-2768-6832-9293",
       "redactedFinding": "πŸ‘€πŸ‘€πŸ‘€πŸ‘€-πŸ‘€πŸ‘€πŸ‘€πŸ‘€-πŸ‘€πŸ‘€πŸ‘€πŸ‘€-πŸ‘€πŸ‘€πŸ‘€πŸ‘€",
        "detector": {
          "name": "Credit Card Number",
          "uuid": "74c1815e-c0c3-4df5-8b1e-6cf98864a454"
        },
        "confidence": "VERY_LIKELY",
        "location": {
          "byteRange": {
            "start": 25,
            "end": 44
          },
          "codepointRange": {
            "start": 25,
            "end": 44
          }
        },
        "redactedLocation": {
          "byteRange": {
            "start": 25,
            "end": 44
          },
          "codepointRange": {
            "start": 25,
            "end": 44
          }
        },
        "matchedDetectionRuleUUIDs": [],
        "matchedDetectionRules": [
          "My Match Rule"
        ]
      }
    ]
  ],
  "redactedPayload": [
    "",
    "",
    "My credit card number is πŸ‘€πŸ‘€πŸ‘€πŸ‘€-πŸ‘€πŸ‘€πŸ‘€πŸ‘€-πŸ‘€πŸ‘€πŸ‘€πŸ‘€-πŸ‘€πŸ‘€πŸ‘€πŸ‘€"
  ]
}

Regular Expression Example

The following example shows a Detection Rule composed of two Detectors defined using regular expressions – one for the format of an International Standard Recording Code (ISRC) and one for the format of an International Standard Musical Work Code (ISWC) – matching either of which will trigger the Detection Rule (by using the logicalOp β€œAny”).

We will provide a payload of two strings, one of which will match the ISRC and one of which will match the ISWC.

curl --location --request POST 'https://api.nightfall.ai/v3/scan' \
--header 'Accept: application/json' \
--header 'Authorization: Bearer NF-rEpLaCeM3w1ThYoUrNiGhTfAlLKeY123' \
--header 'Content-Type: application/json' \
--data-raw '{
     "config": {
          "detectionRules": [
               {
                    "detectors": [
                         {
                              "regex": {
                                   "isCaseSensitive": false,
                                   "pattern": "[A-Z]{2}-?\\w{3}-?\\d{2}-?\\d{5}"
                              },
                              "minNumFindings": 1,
                              "minConfidence": "POSSIBLE",
                              "detectorType": "REGEX",
                              "displayName": "ISRC Code Detector"
                         },
                         {
                              "regex": {
                                   "isCaseSensitive": false,
                                   "pattern": "T-[0-9]{3}\\.[0-9]{3}\\.[0-9]{3}-[0-9]"
                              },
                              "minNumFindings": 1,
                              "minConfidence": "POSSIBLE",
                              "detectorType": "REGEX",
                              "displayName": "ISWC Code Detector"
                         }                         
                    ],
                    "name": "ISRC and ISWC Code Detection Rule",
                    "logicalOp": "ANY"
               }
          ]
     },
     "payload": [
          "Non Matching Payload",
          "US-S1Z-99-00001 is an example ISRC Code: ",
          "The ISWC for Symphony No. 9 is T-905.029.737-5"
     ]
}
'

The returned response demonstrates how findings are returned, with a finding per payload entry and the Detection Rule and Detector that matched the payload, if any.

The byte range that triggered the match is also provided. In the case of the 2nd item in the payload, since the match occurred at the beginning of the string, it has a location where the byteRange start is 0. In the case of the 3rd payload entry the location offset is 31.

{
    "findings": [
        [],
        [
            {
                "finding": "US-S1Z-99-00001",
                "detector": {
                    "name": "ISRC Code Detector",
                    "uuid": "d8be87c9-4b44-41fd-b78c-8d638fe56069"
                },
                "confidence": "LIKELY",
                "location": {
                    "byteRange": {
                        "start": 0,
                        "end": 15
                    },
                    "codepointRange": {
                        "start": 0,
                        "end": 15
                    }
                },
                "matchedDetectionRuleUUIDs": [],
                "matchedDetectionRules": [
                    "ISRC and ISWC Code Detection Rule"
                ]
            }
        ],
        [
            {
                "finding": "T-905.029.737-5",
                "detector": {
                    "name": "ISWC Code Detector",
                    "uuid": "faf4c830-f2ac-4934-bf9c-ff20f5a6f420"
                },
                "confidence": "LIKELY",
                "location": {
                    "byteRange": {
                        "start": 31,
                        "end": 46
                    },
                    "codepointRange": {
                        "start": 31,
                        "end": 46
                    }
                },
                "matchedDetectionRuleUUIDs": [],
                "matchedDetectionRules": [
                    "ISRC and ISWC Code Detection Rule"
                ]
            }
        ]
    ]
}

Word List Example

The following example shows how a word list may be used instead of a regular expression.

curl --location --request POST 'https://api.nightfall.ai/v3/scan' \
--header 'Accept: application/json' \
--header 'Content-Type: application/json' \
--header 'x-api-key: NF-rEpLaCeM3w1ThYoUrNiGhTfAlLKeY123' \
--data-raw '{
     "config": {
          "detectionRules": [
               {
                    "detectors": [
                         {
                              "wordList": {
                                   "values": [
                                        "cat",
                                        "dog",
                                        "rat"
                                   ],
                                   "isCaseSensitive": false
                              },
                              "minNumFindings": 1,
                              "minConfidence": "POSSIBLE",
                              "displayName": "animals",
                              "detectorType": "WORD_LIST"
                         }
                    ],
                    "name": "WordListExamples",
                    "logicalOp": "ANY"
               }
          ]
     },
     "payload": [
          "THE CAT SAT ON THE MAT",
          "The dog and the rat are on the west bank of the river",
          "No one here but use chickens"
     ]
}'

Below is the resulting payload with the findings detected in our different payload strings.

Note that since the isCaseSensitive flag is set to "false" for the detector, so the first string in our payload matches a word from our word list.

Also note that the confidence level for a word list match defaults to "LIKELY" so you should not set a minConfidence level higher than that if you want matches to result.

{
    "findings": [
        [
            {
                "finding": "cat",
                "detector": {
                    "name": "animals",
                    "uuid": "c033e224-034a-417f-9c0d-0c8d13f462bb"
                },
                "confidence": "LIKELY",
                "location": {
                    "byteRange": {
                        "start": 4,
                        "end": 7
                    },
                    "codepointRange": {
                        "start": 4,
                        "end": 7
                    }
                },
                "matchedDetectionRuleUUIDs": [],
                "matchedDetectionRules": [
                    "WordListExamples"
                ]
            }
        ],
        [
            {
                "finding": "dog",
                "detector": {
                    "name": "animals",
                    "uuid": "c033e224-034a-417f-9c0d-0c8d13f462bb"
                },
                "confidence": "LIKELY",
                "location": {
                    "byteRange": {
                        "start": 4,
                        "end": 7
                    },
                    "codepointRange": {
                        "start": 4,
                        "end": 7
                    }
                },
                "matchedDetectionRuleUUIDs": [],
                "matchedDetectionRules": [
                    "WordListExamples"
                ]
            },
            {
                "finding": "rat",
                "detector": {
                    "name": "animals",
                    "uuid": "c033e224-034a-417f-9c0d-0c8d13f462bb"
                },
                "confidence": "LIKELY",
                "location": {
                    "byteRange": {
                        "start": 16,
                        "end": 19
                    },
                    "codepointRange": {
                        "start": 16,
                        "end": 19
                    }
                },
                "matchedDetectionRuleUUIDs": [],
                "matchedDetectionRules": [
                    "WordListExamples"
                ]
            }
        ],
        []
    ],
    "redactedPayload": [
        "",
        "",
        ""
    ]
}
pre-defined Detection Rules
native detectors
extensive library
confidence level
Using Redaction