Custom Detectors
Learn how to create detectors for detecting file types and detecting files through fingerprinting.
You can customize any existing machine learning-based Nightfall detector by adding exclusion rules and/or context rules.
Exclusion rules in Nightfall act essentially as an “allow list,” enabling you to define data that should not trigger a DLP violation.
One common example applies to the Email detector. Many organizations choose to ignore instances of their own email domains.
The exclusion rules help you improve your operational cost by filtering out noise at the source, eliminating unnecessary review of known findings.
RegEx exclusions allow you to define a regular expression with either partial or full matching to exclude what would otherwise be considered a match based on the rule you are customizing.
Dictionary exclusions allow you enter individual terms or upload a comma delimited list of terms that should be excluded from matches. For instance, you may want to exclude a list of of employees from being picked up by the Person Detector.
If you use an uploaded list, this list will become available for use again in other dictionary exclusions you define.
File type exclusions allow you to exclude file matches when the file type matches a particular mime type. This allows you to eliminate matches on certain file types that you are confident are safe.
You can eliminate matches for files that you know generate false positives by taking advantage of Nightfall's file fingerprinting functionality available in our File Fingerprint Detector.
The fingerprinting algorithmically creates a unique identifier for files you upload by mapping the data of the document to a signature that can be recalled quickly. The files you may upload for file exclusions are limited to CSV or TXT files and may up to 25Mb in size per file.
Context Rules tell a detector to analyze the pre- and post- context around a token, looking for hot or cold words to assess the confidence level of the violation (context can uplevel or downlevel the detection confidence).
For example, a company may want to detect occurrences of their account numbers, which are 10-digit numbers. However, there are a lot of 10-digit numbers in the world, and such a broad detection rule could be very noisy. One option is to add context to the rule, such as the appearance of the strings “account” or “acct” near the token. An appearance of the token along with the context will be upweighted to a “very likely” violation, whereas the token without the context would be weighted as “possible.”
Context rules will typically be applied to custom detectors, as Nightfall’s machine learning detectors already take context into account. If you have a specific use case and want to understand the specific context analyzed by each detector, our team will be happy to speak with you.
Short for regular expression, a regex is a string of text that allows you to create patterns that help match, locate, and manage text. If you want to look for a common pattern, you can create your own custom regex to search for that specific pattern. If a token is matched, Nightfall will alert with a confidence level of ‘Likely’.
Nightfall allows you to create your own custom Detectors from a list of tokens using the dictionary Detector type. This allows you to manage a large list of keywords or phrases for which to scan. You may make use of existing sensitive data lists.
Dictionary Detectors are created by uploading a text file up to 25 MB in size. Each line will represent one entry.
Nightfall enables you to discover the location of specific files that you have deemed sensitive and want to avoid sharing.
This discovery is done through document fingerprinting. Fingerprinting is the process of algorithmically creating a unique identifier for a file by mapping the data of the document to a signature that can be recalled quickly. This allows the file to be identified in a manner akin to how human fingerprints uniquely identify individual people.
This functionality is achieved in Nightfall by creating a specific Detector type called a File Fingerprint Detector.
The Fingerprint Detector allows you to create a fingerprint for one more files (a sort “handful” of fingerprints, if you would).
To create a Fingerprint Detector, select “Detectors” from the left hand navigation and click the button labeled “+New Detector” in the upper right hand corner. From there a drop down list of Detector types will be displayed which will include the “Fingerprint” Detector type.
When you create a File Fingerprint Detector you can upload up to 50 files that need to be fingerprinted. The file size limit is 25MB.
Once the fingerprint is generated, the actual content of the file is discarded so no sensitive content is stored on Nightfall’s system.
Note that you can not update Fingerprint Detectors, so any modification to the original file or underlying requires that you create a brand-new Fingerprint Detector.
You may then treat the Fingerprint detector like any other Detector and incorporate it into a Detection Rule using its unique Detector identifier.
You may incorporate these Detectors into Policies that will alert you whenever files that match the fingerprint are detected.
Nightfall’s File Type detection allows you to implement compliance policies that detect and alert you when particular file types that are not allowed in a given location are discovered.
This functionality is implemented by creating a specific Detector called a “File Type Detector”
To create a File Type Detector, select “Detectors” from the left hand navigation and click the button labeled “+New Detector” in the upper right hand corner. From there a drop down list of Detector types will be displayed which will include the “File Type” Detector type.
You will then select one or more file types for which to scan by selecting from a list of mime-types.
You can either scroll through the list of mime-types in the select box or you may type in a portion of the mime-type and the contents of the select box will be filtered to match your input.
Nightfall supports detection for a wide variety of mime-types. See the Internet Assigned Numbers Authority’s (IANA) website for a definitive list of mime-types. Note however that Nightfall does not support the detection of audio and video related mime-types.
Detection of file types is done based on the file contents, not its extension.
File Type Detectors vary from other Nightfall Detectors in that the attributes of
scope and confidence are not relevant to File Type DetectorsOnce you have added all the mime-types you wish to scan for, save your new Detector.