September 2024

This section enlists the feature enhancements released by Nightfall in September 2024.

• Ability to Search Within Google Drive Event Attributes: Nightfall now allows you to search Google Drive Events by using the values from Google Drive #event-detail-view. You can now search and filter Google Drive Events with Google Drive shared drive name, Link settings, Shared with internal users & groups, Shared with external users & Group, File owner, and Last edited by attributes. Click here to learn more about this feature.

• Introduction of Events and Policies in Data Encryption (early preview): Nightfall has now introduced Events and policies in Data Encryption. You can now create policies in Data encryption and view Event data for these policies. Click here to learn more about this feature.

• Sender, Recipient, and Domain Filtering in Email DLP: Nightfall now allows you to configure Email DLP policies to include or exclude specific senders, recipients, and domains for monitoring. Click here to learn more about this feature.

• Support for Okta as an Identity provider (early preview): Nightfall now allows you to perform Directory Sync by using Okta as an Identity provider. Click here to learn more about this feature.

• Slack User and User Group Filtering (early preview): Nightall now allows you to configure Slack policies by filtering specific users and user groups. Click here to learn more about this feature.

• Detector for Philippines Tax ID Number (early preview): Nightfall has now introduced a detector that can detect the TIN (tax ID number) in Philippines. Click here to learn more about this feature.

• Improvements to the PHI Detector: Nightfall has now enhanced the PHI detector. This enhancement ensures improved data output handlings, greater accuracy in file classification, greater model precisions, and validation of findings. Click here to learn more about this feature.

August 2024

This section enlists the feature enhancements released by Nightfall in August 2024.

• Detectors for Southeast Asian Countries: Nightfall has released new detectors which support sensitive data protection for southeast asian countries namely Cambodia, Indonesia, Philippines, Singapore, and Vietnam. Click #standard-pii-apac to learn more about the detectors.

• Risk Scoring: Nightfall now allows you to rank violations in accordance to the risk associated with them. Click here to learn more about this feature.

• New Vendor Specific API Key Detectors: Nightfall has launched vendor specific API key detectors for four vendors namely OPenAI, Pagerduty, Databricks, and, Hugging Face. Click here to learn more about the detectors.

• Introduction of Exfiltration Prevention for Salesforce (early access): Nightfall now supports exfiltration prevention for Salesforce. Click here to learn more about this feature.

• Enhanced Image ID Detectors: Nightfall has significantly enhanced the capabilities of image ID detectors. These enhancements reduce false positives and improve detector coverage. Click here to learn more about the detectors.

• Unified Events Page: Nightfall has now unified all the events across all the products on a single page. Going forward, Violations would now be known as Events. Click here to learn more about the feature.

• Ability to Exclude Applications from Google Drive Exfiltration: Nightfall now allows you to exclude applications from being monitored in Google Drive exfiltration policies. Click here to learn more about the feature.

• Introduction of Lineage-Based Policies in macOS (early preview): Nightfall has now introduced endpoint lineage and lineage based policies for macOS. Click here to learn more about the feature.

July 2024

This section enlists the feature enhancements released by Nightfall in July 2024.

• Introduction of Data Exfiltration and Posture Management for Google Drive: The Google Drive integration is now equipped with exfiltration prevention and posture management capabilities. For more information, refer to this document for exfiltration prevention and this document for posture management.

• IDP Integration with Corporate Directory Services: Nightfall now supports directory service in Microsoft Entra and Google Directory. Click here to learn more about this feature.

• Introduction to Nightfall Data Encryption: Nightfall has now introduced data encryption for the Gmail integration. Click here to learn more about this feature.

• Sensitive Data Protection to Emails: Nightfall can now scan your emails for sensitive data. This feature is currently available in Gmail. Click here to learn more about this feature.

• Introduction of AI-powered Automated Labelling in Google Drive (early preview): Nightfall now allows you to use Labels in Google Drive policies. Click here to learn more about this feature.

• Introduction of Role Based Access Control (RBAC): Nightfall now supports RBAC for better resource management. Click here to learn more about this feature.

• Improved Navigation Bar: The Nightfall application's navigation bar is now enhanced for better user experience. You can view the latest navigation bar by logging in to Nightfall.

June 2024

This section enlists the feature enhancements released by Nightfall in June 2024.

• Sensitive Data Protection now available for OneDrive: Nightfall now supports sensitive data protection in Microsoft OneDrive for Business. Click here to learn more about this feature.

• Sensitive Data Protection now available for Teams Chat: Nightfall now supports protection of sensitive data in Microsoft Teams for chat messages. Click here to learn more about this feature.

• Bulk Remediation for Google Drive Violations: Nightfall now supports bulk remediation for violations reported in Google Drive. Click here to learn more about this feature.

• Ability to Detect UK and Canadian Street Addresses: Nightfall has enhanced the Street_Address detector to detect UK and Canadian street addresses. Click here to learn more about this detector.

• Ability to Detect UK Phone Numbers: Nightfall has enhanced the Phone_number detector to detect UK and Canadian street addresses. Click here to learn more about this detector.

• Exfiltration Protection for macOS Devices (early access): Nightfall now supports exfiltration prevention in macOS devices.

May 2024

This section enlists the feature enhancements released by Nightfall in May 2024.

• Launch of Firewall for AI: The rebranded and retargeted version of Nightfall Developer platform is now available as Firewall for AI, which focuses on Generative AI model building. Click here to learn more about this feature.

• Nightfall Data Exfiltration for Google Drive (early preview): Nightfall has launched exfiltration prevention platform. Google Drive is the first tool that is currently supported by Nightfall for exfiltration prevention. Click here to learn more about this feature.

• Sensitive Data Protection for Gmail: Nightfall now provides security against sensitive data exposure through emails; specifically Gmail. Click here to learn more about this feature.

• Ability to Include and Exclude Drives in Google Drive Policies: Nightfall now provides you the flexibility to include or exclude specific drives from being scanned. you can configure these settings while creating a Google Drive policy. Click here to learn more about this feature.

• Updates to ID document Classifiers: Nightfall has now updated the detectors that pick up shared drivers licenses, passports, credit cards, and us social security cards. Click here to learn more about this detector.

April 2024

This section enlists the feature enhancements released by Nightfall in April 2024.

• Nightfall Sensitive Data Protection (early access): Nightfall sensitive data protection for emails is now available for early access. With this feature, you can protect sensitive data in your emails sent through Gmail. Click here to learn more about this feature.

• In-product Announcements: Nightfall has now introduced in-product announcements. You can click the question mark icon a the bottom right of your screen from any Nightfall page and select the News option to view the latest announcements.

• Support for directory Sync via Identity Providers (early preview): Nightfall now supports the Directory Sync feature. This feature is available in early access. With this feature, you can now import users and user groups via identity providers like Google Directory and Microsoft Entra ID. Click here to learn more about this feature.

March 2024

This section enlists the feature enhancements released by Nightfall in March 2024.

• Introduction to GitHub Deduplication feature: Nightfall has introduced the deduplication feature for GitHub integration. This feature automatically detects and prevents duplicate entries of GitHub violations on the Violations page. You can refer to this document to learn more about this feature.

February 2024

This section enlists the feature enhancements released by Nightfall in February 2024.

• Introduction of APIs to detect sensitive data: Nightfall has now introduced APIs to detect sensitive data in SaaS applications. You can use these APIs to You can use these APIs to fetch violations, search through violations, annotate findings, and so on. You can navigate to this link for API documentation for SaaS applications.

• Introduction of the Independent ML Services: Nightfall now uses independent Machine Learning services to enable Real-Time File Scanning.

• Enhanced Database Connection String Detector: The Database Connection string detector is now updated. This detector can now recognise URI-based, driver-based, and key-value pair based connection strings. Click here to learn more about this detector.

• Enhancements to the Password Detector: The Password detector is now enhanced to recognise commonly used password placeholders in code. These placeholders are no longer identified as violations by the detector, thus reducing the false positive cases. Click here to learn more about this detector.

• New "Ignore All" Option for Google Drive Violations: Nightfall now allows you to ignore all the current and future Violations arising out from a specific file in your Google Drive. Click here (refer to step 5) to learn more about this feature.

January 2024

This section enlists the feature enhancements released by Nightfall in January 2024.

• Improvements to the Person Names Detector: The Person Names detector is now enhanced to recognize synthetic names from Brazil, Portugal, Mexico, Argentina, Spain, and Colombia countries.

• New Pending and Resolved Tabs in Violation Management: The Violations page now has two new tabs, Pending and Resolved. The Pending tab displays the list of Violations to be resolved and the Resolved tab displays the list of Resolved and Ignored Violations.

• Improvements to the Street Address Detector: The street address detector is now enhanced to identify Spanish and Portuguese street addresses, resulting in enhanced address detection.

• Improvements to the Password Detector: The Password detector is now updated to recognize and ignore commonly used development environment passwords, environment variables, and encrypted passwords.

• New Ignore All Option for Google Drive Violations: Nightfall now allows you to ignore all the current and future Violations arising out from a specific file in your Google Drive.

• Automatic Annotation and Auto Resolution of Findings: Nightfall now allows you to automatically annotate all the findings that are similar to a Finding. With this feature, if all the Findings of a Violation are false positive the Violation is automatically ignored.

• Introduction of Custom Date Range Filters: Previously, you could filter historic Dashboard and Violation data using fixed time frames like Last 7 Days, Last 30 Days, and so on. Nightfall now allows you to set custom date range filters on the Dashboard and Violations pages.

The International Organization for Standardization (ISO) is a non-governmental organization that provides transnational standards across technology and engineering fields. ISO 27001 is a set of standards or ISMS (information security management system) to help orgs build out a coordinated and functional infosec program by assessing risk and identifying the types of controls critical to securing data in electronic systems.

There are currently two versions of ISO 27001

Version 2013

This version of ISO 27001 can no longer be used after October 31, 2025. It has limited data protection requirements that are outlined in a control supplement called ISO 27002.

These two requirements would be supported with Nightfall:

• 11.2.6: Security of Equipment & Assets Off-Premises

• Framing: data is the asset, and the cloud / SaaS is considered off-premises

• Security controls need to be applied to off-site assets, which essentially means protecting data (the asset) on the cloud (the off-site / third-party data center). ISO requires the same technical controls as if it were on-site, meaning that Nightfall supports these ISO control sections:

  • 8.1.3 / 8.2.3: Acceptable use of information and systems - this is the appropriate handling (i.e., sharing / disclosure) of sensitive data, which is Nightfall’s main mission

  • 12.4.1 - 12.4.3: Monitoring of sensitive data use

  • 16.1.2 - 16.1.3: Information Security Event Reporting

  • 18.2.2 - 18.2.3: Compliance with policies and standards for information security

• 13.2.1: Information Transfer Policies & Procedures

• Communications must be secure, regardless of which system used. With Slack, Google Workspace, and other systems in development, Nightfall provides protection of comms over public internet to a variety of audiences, and is thus a control that supports this ISO requirement for confidentiality by taking into account the type, nature, amount and sensitivity or classification of the information being transferred.

Version 2022

This latest version of ISO 27001 will be required after October 31, 2025, and it adds the following new data protection requirements:

• A.8.12: Data leakage prevention. A tool like Nightfall is now required if processing sensitive information, which is almost always the case. The requirement states that “data leakage prevention measures shall be applied to systems, networks, and any other devices that process, store or transmit sensitive information.”

• In support of the above requirement, Nightfall also would be a control for:

  • A.8.10: Information deletion. Nightfall’s automated remediation such as deletion enabled this requirement, which states that “information stored on information systems, devices or in any other storage media shall be deleted when no longer required.”

  • A.8.11: Data masking. Nightfall’s data masking in protecting data is identified as a specific requirement. The requirement states “data masking shall be used in accordance with the organization’s topic-specific policy on access control and other related topic-specific policies, and business requirements, taking applicable legislation into consideration.”

  • A.8.16 Monitoring activities: networks, systems and applications shall be monitored for anomalous behavior and appropriate action taken to evaluate potential information security incidents.

• A.8.28: Secure coding. Nightfall’s protection of secrets and keys, none of which should ever be disclosed in development, supports this ISO requirement, which states that “secure coding principles shall be applied to software development.”

Nightfall is:

• Agentless. Nightfall isn't deployed as software that requires installation, rather it integrates with the applications we secure through APIs. This makes deployment easy and updates to our platform effortless, without getting end-users or IT involved.

• API driven. Central to Nightfall is the API driven nature of our platform. Connecting with cloud platforms via API means that visibility and security policies immediately apply at the application layer. Nightfall can derive platform-specific context & metadata, as well as provide granular, platform-specific actions, versus broad-brush blocking on the network.

• Agnostic. Nightfall is platform, endpoint, and network agnostic in that we’re capable of integrating with cloud platforms quickly and can provide single pane of glass visibility across multiple cloud apps simultaneously. Via our Developer platform, you can also provide coverage for applications we don't natively integrate with, including your own custom apps be they cloud-based or on-prem.

• Automated. Nightfall doesn't just provide visibility into the cloud, but helps automate policies whenever possible. The sheer volume of data that moves through cloud systems combined with the always-on nature of cloud applications means that incidents can happen at any time and will require immediate remediation. Automation ensures that security teams can respond to these as quickly as possible.

• Accurate. Finally, in order to help security teams process the massive amounts of data in the cloud, cloud-native DLP must be accurate. The accuracy of Nightfall is enabled by the same systems that allow us to automate detection — an effective use of machine learning that can quickly and accurately identify when business-critical data has been exposed.

See the diagram below to understand the coverage provided by Nightfall.

In today's data-centric environment, organizations are moving to the cloud to cut down infrastructure expenses and concentrate only on their organization tasks thus empowering them to build powerful applications catering to their customers.

When organizations migrate to the cloud, their entire infrastructure is stored on the cloud. The organization's infrastructure moving to the cloud implies that its sensitive data is also stored in the cloud. Sensitive information being stored in the cloud is becoming a normal phenomenon.

conducted by Thales on 3000 IT professionals across 18 countries revealed eye-opening facts. A whopping 75% of respondents said that around 40% of their data stored in the cloud is sensitive. With Cloud being the go-to option for modern businesses, the amount of sensitive information moving to the cloud will only increase in the coming years.

So, What's the Harm of Sensitive Data Residing in the Cloud?

The sensitive data stored on the cloud can either belong to the organization itself or it can be the customer's data required for business purposes. If this sensitive data is lost or even exposed to unauthorized users the consequences can be fatal for organizations and their customers. It's the responsibility of the organization operating in the Cloud to ensure the safety of customers and their sensitive data. Governments around the globe have defined complaint frameworks like GDPR, PII, HIPAA, and so on, which must be adhered to by cloud organizations that collect customer data for operations.

What Can Pose a Threat to Sensitive Data?

Data breach is a well-known threat that has already affected many businesses around the world and is continuing to do so, bringing companies to a standstill and in the worst cases even closure of businesses.

A data breach is a scenario in which your organization's data security is compromised by a malicious user or group (hacker), who steals your organization's sensitive data. Once the data is stolen, hackers can demand ransom to return your data which can lead to data loss by your organization. Hackers might even directly expose your sensitive data on platforms like the dark web without demanding any ransom.

How Can I Prevent Data Breach?

As we saw in the previous section, a data breach can easily lead to data loss or data exposure. So, you need to prevent data breaches in your organization. But what exactly can cause a data breach?

The revealed that human error is the most common cause that leads to a data breach. Of all the data breaches caused, human error was responsible for 55% of the total data breaches. (The next distant factor was the exploitation of vulnerabilities which accounted for 21% of all the data breaches).

A human error is a scenario in which an employee from the cloud provider organization leaks out sensitive data unintentionally. This is known as a data leak. Data leaks are pretty common in organizations because employees are generally occupied in their day-to-day tasks. Some or most of their tasks involve the usage of sensitive data present in the organization. While using sensitive data there is a very high possibility that employees might leak it publicly, causing a data leak, thus leaving room for a potential data breach attack which can ultimately lead to data loss.

The following examples are scenarios of data leaks that can be caused by employees.

• A developer commits a piece of code to GitHub that consists of an API key or some other credentials.

• An employee shares an image in a public Slack channel that contains sensitive data.

• A developer submits live API keys to ChatGPT to generate a block of code.

• An employee uploads a document with sensitive data to a public Google Drive or AWS S3 bucket.

• A support team member reveals secret data in a Zendesk ticket.

The above examples are pretty common cases of data leaks which are difficult for any organization to prevent. In the real world, many more scenarios of data leaks go unnoticed until they lead to a data breach. Even after a data breach, it could be difficult for organizations to figure out how hackers were able to gain access to sensitive data, which can help them stop such attacks in the future. It is only when organizations perform a hardcore root cause analysis they get to know that a minor data leak by an employee led to a mammoth data breach.

So, it's pretty clear that not exercising anti-data leak solutions in your organization is as good as serving your organization's sensitive data on a platter to malicious attackers. This is because some employee at some point is bound to accidentally cause a data leak.

How Can I Prevent a Data Leak?

Framing policies to protect sensitive data and educating employees about these policies is a common approach followed by organizations. However, every organization that experienced a data leak, did have data protection policies and implemented rigorous training to employees on adhering to these policies which unfortunately could not prevent the attack.

Another approach can be the use of Cloud Access Security Brokers (CASBs), or some data leak prevention (DLP) tools that can automatically halt unintentional data leak attempts by employees. However, the issue with such tools is that many of them cannot be used in the cloud. Some other DLPs are deployed as agents. The issue with these agents is that when information is transferred to cloud applications from unmanaged or off-network devices, these legacy solutions are powerless to intercept it. Once the sensitive information is stored within the cloud application or infrastructure, legacy endpoints, and network solutions can no longer see it. As a result, users of legacy DLP solutions are left with no visibility into sensitive data that already exists in the cloud, or which is being transferred to the cloud on unmanaged networks or devices.

So, how do you protect your sensitive data in the cloud?

Nightfall - AI Native Cloud DLP

Nightfall’s Cloud DLP provides a solution to this problem. Nightfall is cloud-native and integrates directly with other cloud applications and infrastructure at the application level via API. Nightfall can inspect content stored within the cloud application regardless of how it got there, for complete visibility into cloud DLP risk. Another key advantage of direct cloud-native integration is that Nightfall can take remediation actions on sensitive data that is discovered in the cloud, thus eliminating the DLP risk at the source - a method that legacy solutions cannot hope to achieve. Nightfall’s cloud-native DLP does not require the installation of agents and can be integrated with your cloud applications in just a few clicks. The result is a DLP solution for the modern world that can proactively identify and eliminate DLP risk across your cloud environment.

Nightfall Product Updates 2022

Nightfall Product Updates 2021

Unlike traditional IT environments, cloud systems have no . Historically, security revolved around keeping intruders out and hardening systems explicitly owned by an organization. However, the entire point of cloud adoption is to enable data to be wherever it needs to in order to be useful. As such, data itself is best thought of as part of your organization’s attack surface — the more data you have, the bigger your data exposure risk.

1. The cost of exposures tends to be higher in the cloud

2. Security and IT teams are stretched thin

3. It’s very difficult to consistently enforce proper data policies in the cloud

One of the key problems organizations face regarding security and compliance is ensuring that employees are aware of best practices and verifying that they’re following these guidelines. Without sufficient visibility into cloud systems, this can be very difficult to do for the reasons we’ve highlighted above.

4. The cloud shared responsibility model requires it

Nightfall has several key benefits, including:

• Reductions in both the cost and occurrence of data breaches. Nightfall limits the exposure of things like credentials and secrets, which are a common cause of data breaches. Nightfall also limits the exposure of PII and other business critical content, which means that if a breach does occur, its impact is minimized

• Nightfall generates time savings by helping automate the task of discovering, classifying, and appropriately remediating sensitive and business-critical data.

To learn more, see our ROI calculator:

A common misconception is that Cloud Access Security Brokers (CASBs) can accomplish the same level of DLP protection as cloud-native DLP solutions. CASBs are deployed as proxies and sit between the network layer and the cloud application layer allowing them to inspect traffic traveling between your network and designated cloud environments. The limitations of CASBs stem primarily from the fact that they can only inspect network traffic, and do not natively integrate with cloud applications or infrastructure.

A summary of differences between Nightfall’s Cloud DLP and CASBs is shown below.

Many companies seek to become SOC 2 compliant to showcase their strong security practices for safeguarding their company’s and their customer’s data.

During SOC 2 compliance audit periods, auditors review that security controls are in place, triggered, and responded to appropriately to assess the strength of an organization’s security posture. Auditors generally look for a number of controls, including the following:

• Information security practices that are documented & managed on an ongoing basis

• Data classification policies and protocols that detail the security and handling of customer data

Checklist for meeting SOC 2 controls with Nightfall

Nightfall’s Customer Success team specializes in helping you leverage the platform to meet your compliance needs successfully.

1. Configure detection rules that detect sensitive data your business handles. Select from common templates in our library for out of the box coverage.

2. Enable real-time monitoring on business applications that house sensitive data such as Slack, Google Drive, Confluence, Jira, and GitHub.

3. Implement manual or automated workflows and processes to remediate any findings.

4. Run historical scans to search for sensitive data that exists in data silos today.

5. Visualize historical scan results in a custom Nightfall dashboard.

6. Engage Nightfall’s Managed Services team to facilitate bulk remediation of sensitive data at rest in cloud silos.

7. Review and export scan results should they be required in the event of an audit.

The following table lists security practices and policies that companies should implement for a strong security posture. A data classification & protection platform like Nightfall can help companies enforce and manage these controls.

Nightfall detection engine looks for sensitive data. When sensitive data is detected, a notification or automated remediation action is initiated. The detection engine consists of three components; Detectors, Detection Rules, and Policies.

A Detector is an entity that can detect the presence of sensitive data in publicly exposed entities. You can either use the default detectors provided by Nightfall or create your custom detectors.

A Detection rule is a group of detectors. Once you create your detectors or finalise the Nightfall detectors you wish to use, you can add them to a detection rule. You can customize the degree of weightage for each detector in a detection rule.

Once you finalize the creation of detection rules, you can add them to policies. A Policy consists of multiple detection rules. When a detector in any of the detection rules detects sensitive data, the Policy can trigger auto notifications to be sent to specific users, initiate automated remediation measures, and allow end-users to take remediation actions.

So, a Detector or a Detection rule do not have the ability to trigger alerts or take remediation steps by themselves, you need to add them to a policy.

The following diagram shows how you can leverage the three entities of the Nightfall Detection engine.

Nightfall is a SaaS-based cloud application. This document explains how you can sign up for a Nightfall account and start using the Nightfall application.

To sign up for Nightfall:

1. Navigate to .

2. Click Sign up.

1. Provide the required details and click Continue.

1. (Optional) Enter the additional details.

2. Click Create account.

The following verification screen is displayed. Check your Email. You should have received a new Email from support@nightfall.ai. Click the verification link in the Email to verify your account.

1. Click Continue to login to navigate to the login screen. You can log in to Nightfall only after verifing your Email.

The blank Nightfall Dashboard page is displayed as follows.

Nightfall SaaS and Firewall for AI

The Nightfall SaaS app provides you with numerous integrations to connect to various third-party applications in which your data resides. You can connect to the integration in which your organization's data resides and then start scanning your data with Nightfall to detect sensitive information.

By default, none of the integrations are active. You must contact Nightfall to subscribe to a paid plan and activate the required integration.

If an integration is not available for your desired application, you can subscribe to the Nightfall Firewall for AI. The Firewall for AI's APIs and SDKs help you integrate even those applications for which integration is not available in the Nightfall app.

Payment card industry (PCI) compliance is mandated by credit card companies to help ensure the security of credit card transactions in the payments industry. Payment card industry compliance refers to the technical and operational standards that businesses follow to secure and protect credit card data provided by cardholders and transmitted through card processing transactions.

Financial data, especially credit card numbers, pose an obvious DLP risk. Banks, financial institutions, and others concerned with protecting financial data typically use Nightfall’s payment card detectors.

PCI DSS Version 3.2.1

Appendix 3.2.6 states the requirement for DLP applicable to ALL PCI-covered entities, making this the strongest case for Nightfall. As written:

• Appendix 3.2.6 - Mechanisms are implemented for detecting and preventing clear text PAN from leaving the cardholder environment.

• Note that PCI’s example given explicitly states DLP: “Mechanisms to detect and prevent unauthorized loss of cleartext PAN [credit card number] may include the use of appropriate tools such as data loss prevention (DLP) solutions as well as manual processes and procedures.”

The following PCI DSS sections further supported the need for a DLP:

3. Protect Stored account Data. specified that cardholder data must be protected at rest. Nightfall can identify when such data is being stored somewhere where unauthorized access has or may occur.

7. Restrict access to system components and cardholder data by business need to know. Same as above point. Nightfall can determine when need-to-know access might be violated.

10. Log and monitor all access to system components and cardholder data. Same as above, while Nightfall is not an access control, within certain integrations Nightfall can help determine who has had access to a specific piece of data.

12. Support information security with organizational policies and programs. Nightfall can help teams build enforceable information security policies.

Please find our template of detectors for PCI compliance here:

A Detector is a tool that scans any online entity to detect if any sensitive data is present in the entity. An "entity" can refer to any resource on the Internet like a file in Google Drive, ticket data in JIRA, code snippets in GitHub, customer data in Salesforce, and so on.

There are two types of detectors; Nightfall Detectors and Custom Detectors.

Nightfall Detectors

Nightfall provides core machine learning-powered detectors known as Nightfall detectors. All the detectors provided by Nightfall, are provided out-of-the-box and are present in your Nightfall environment by default. You can navigate to the Detectors section in the left menu and select the BUILT BY NIGHTFALL option to view all the Detectors provided by Nightfall.

Each Nightfall detector is designed to identify a specific type of sensitive data. For instance, The Email detector can identify Email IDs. You can use this detector to check if any Email IDs from your organization are being leaked out on the Internet.

Similarly, there are detectors to detect personally identifiable information (PII) from various countries. The PII of each country is unique. For instance, French passport numbers have a different format as compared to German passport numbers. So, if you are a French company that wishes to detect if any of the French passport numbers of your employees or customers are leaked or are prone to be leaked, you can use the France Passport detector. Similarly, German companies can use the Germany Passport detector.

The detectors are classified in terms of the sensitive data that they protect. The various categories under which Nightfall detectors are classified are as follows.

• Common Entities: This category consists of detectors that uniquely identify a person. The detectors in this category are Name, Email, Address, and Phone number.

• Content Moderation: This category consists of detectors that can identify harmful data in images. The detectors in this category are Violent or Disturbing Image detector which can detect violent content from images and Sexually Explicit Image detector which can identify sexual activity in images.

• Documents: This category consists of detectors that can identify sensitive data in images. Some of the detectors in this category are Credit Card image detector which can detect credit card numbers from images and Passport or Visa Image detector which can detect passport numbers or visa numbers from images.

• Finance - Banking: This category consists of detectors that can identify sensitive data related to the financial industry. Some of the detectors in this category are Canada Bank Account Number detector which can detect bank account numbers from Canada and the IBAN code detector which can detect the IBAN codes.

• Financed - PCI: This category consists of detectors that can identify sensitive data related to payment cards. Credit card numbers is one such detector that falls under this category and identifies the credit card numbers of customers.

• Network/Hardware: This category consists of detectors that can identify sensitive data related to hardware and networking. The IP address detector in this category can identify IP addresses.

• Healthcare: This category consists of detectors that can identify sensitive data related to the healthcare industry. The Protected Health Information (PHI) detector can identify data that can uniquely recognize a person.

• Personally Identifiable Information (PII) - Americas: This category consists of detectors that can identify sensitive data that can uniquely identify citizens from various countries of the American continent. Some of the detectors in this category are the Canada Driver License Number detector which can identify driving license numbers issued to Canadian citizens and residents, US Driver license detector which which can identify driving license numbers issued to US citizens and residents.

• Personally Identifiable Information (PII) - EMEA/AP: This category consists of detectors that can identify sensitive data that can uniquely identify citizens from various countries in Europe and Asia Pacific. The Indian Aadhar Card number detector is a part of this category that can identify the Aadhar numbers of Indian citizens.

• Secrets: This category consists of detectors that can identify user's sensitive data. The API key detector can identify live API keys exposed to the Internet.

Other detectors that exist are not recommended for use for the above compliance frameworks. For all use cases, Nightfall further recommends:

• Tune and amend Minimum Confidence over time in accordance with your violations and data set

• Scoping should cover all locations where the sensitive data should not be disclosed

• Using Exclusion Rules to reduce false positives and fine-tune alerts

• Reporting false positives for machine learning training to support@nightfall.ai

Nightfall recommends you to read the document and then proceed with this document.

Nightfall is the industry’s first AI-native data leak protection platform. Nightfall integrates directly with cloud apps and data infrastructure to discover, classify, and protect sensitive information. Typical use cases for Nightfall include data loss prevention (DLP), data classification, and content moderation. Nightfall has pre-built integrations for popular cloud apps like Slack, GitHub, Google Drive, Confluence, Jira and Salesforce; and also offers the ability to integrate with any application or data flow via our Developer Platform.

With Nightfall:

• Identify where you have risks related to sensitive information exposure in the cloud.

• Protect sensitive information, for example by removing it from where it doesn’t belong.

• Add content inspection and remediation capabilities to any third-party application, without agents or proxies.

• Leverage machine learning (ML)-based detection for smarter results, with high accuracy, easy tuning, and fewer false positives.

• Configure and customize data detection policies that meet your organization’s unique needs, such as allowing data to exist in one location but not another.

Nightfall provides 150+ ML-based detectors out of the box to identify and remove a wide range of sensitive data, helping organizations to implement a holistic approach to data stewardship across their cloud ecosystem, and to comply with SOC 2, HIPAA, PCI, CCPA, and client requirements.

Nightfall is backed by Bain Capital Ventures, Venrock, Webb Investment Network, and a cadre of high-profile operators, including CEO/executives of Okta, Splunk, FireEye, Atlassian, and Salesforce.

Learn more about Nightfall .

This article discusses how Nightfall, the AI-powered data leak prevention (DLP) platform for SaaS and cloud, can help organizations maintain compliance with various global compliance frameworks across different industries. It provides an overview of Nightfall detection and its pre-built detectors. It then delves into specific compliance frameworks such as GDPR, CCPA, HIPAA, HITRUST, PCI, GLBA/FCRA, SOX, ISO 27001, FERPA, CMMC, and FedRAMP/NIST 800-53, and explains how Nightfall can help organizations comply with the specific requirements of each framework. The article also includes links to additional resources for each compliance framework.

Please note that while these specific frameworks and controls may be relevant to DLP, it is essential to consult the full text of these frameworks and any applicable regulations or guidelines to ensure comprehensive compliance with the law. The information presented here should not be considered legal advice or a confirmation of compliance with any specific laws or regulations. Legal advice specific to your situation is recommended to understand the complete obligations and requirements of these frameworks.

Nightfall Detection

Nightfall comes with many pre-built detectors out of the box. You also have the ability to add custom detectors, rules, keywords, and regexes. To explore our pre-built detectors, visit our .

Data Privacy

GDPR

The General Data Protection Regulation (GDPR) operates under the principle of data protection by design and by default. This requires organizations to implement appropriate technical and organizational measures to ensure the protection of personal data. Specifically, Data Leak Prevention (DLP) solutions can be used to detect and prevent unauthorized transmission or storage of personal data, such as names, addresses, and personal identification numbers.

The specific controls are:

• Article 32: Security of processing, which requires organizations to implement appropriate technical and organizational measures to ensure a level of security appropriate to the risk. Nightfall is a control to protect personal data from unauthorized access, alteration, destruction, or accidental loss.

• Article 5(1)(f): Data minimization, which requires organizations to limit the collection and retention of personal data to what is necessary for the specific purposes for which it is processed. Nightfall’s realtime monitoring can help organizations to identify and prevent the retention of unnecessary personal data.

• Article 33: Notification of a personal data breach to the supervisory authority, which requires organizations to report personal data breaches to the appropriate supervisory authority without undue delay and, where feasible, not later than 72 hours after having become aware of it, unless the personal data breach is unlikely to result in a risk to the rights and freedoms of natural persons. Nightfall’s realtime monitoring enables organizations to detect and respond to personal data breaches in a timely manner.

• Article 35: Data protection impact assessment (DPIA), which requires organizations to carry out a DPIA when the processing is likely to result in a high risk to the rights and freedoms of natural persons. Nightfall helps organizations to identify and mitigate risks to personal data by detecting and preventing data breaches.

CCPA

The California Consumer Privacy Act (CCPA), much like GDPR, requires protecting personal information via reasonable security measures like DLP. Nightfall is a control to detect and prevent the unauthorized transmission or storage of personal information, such as names, addresses, and personal identification numbers.

These specific CCPA sections and controls supported by Nightfall are:

• Section 1798.81.5 of CCPA requires businesses to implement and maintain reasonable security procedures and practices appropriate to the nature of the information to protect personal information from unauthorized access, destruction, use, modification, or disclosure. Nightfall is a control to protect personal data from unauthorized access, alteration, destruction, or accidental loss.

• Sections 1798.82 and 1798.83 of CCPA requires businesses to disclose to the California attorney general and California residents of any breach of the security of the system data, including a description of the categories and specific pieces of personal information that were or are reasonably believed to have been the subject of a breach. Nightfall helps organizations to identify and mitigate risks to personal data by detecting and preventing data breaches.

• Section 1798.125 of CCPA requires businesses to provide certain rights to California residents regarding their personal information, such as the right to know what personal information is being collected, the right to request deletion of personal information, and the right to opt-out of the sale of personal information. Nightfall helps organizations to identify and control the collection and sharing of personal information.

LGPD

The Lei Geral de Proteção de Dados (LGPD) is the Brazilian data protection law that regulates the processing of personal data. Here are some specific controls and section numbers of the LGPD that are relevant to data leak prevention (DLP) for SaaS:

1. Article 46: This article establishes that the data controller is responsible for adopting security measures to protect personal data against unauthorized access, accidental or unlawful destruction, loss, alteration, communication, or any form of improper or unlawful processing. Nightfall is a control to protect personal data from unauthorized access, alteration, destruction, or accidental loss.

2. Article 47: This article states that the data controller must adopt administrative, technical, and security measures to protect personal data, considering the nature, scope, context, and purposes of the processing, as well as the risks and rights involved. Nightfall is a technical security measure for protecting personal data.

3. Article 48-A: This article, introduced by Provisional Measure No. 869/2018, requires the data controller to carry out a Data Protection Impact Assessment (DPIA) for processing operations that may present risks to individuals' rights and freedoms. Nightfall can conduct historical scans that identify sensitive data like PII across SaaS applications.

4. Article 48-B: Also introduced by Provisional Measure No. 869/2018, this article establishes that the data controller must adopt mechanisms for auditing and demonstrating compliance with data protection regulations. Nightfall can conduct historical scans that provide reports on sensitive data exposure across SaaS applications.

5. Article 48-C: This article, introduced by Provisional Measure No. 869/2018, requires the data controller to implement administrative and technical measures to protect personal data, including measures aimed at preventing unauthorized access, accidental or unlawful destruction, loss, alteration, communication, or any form of improper or unlawful processing. Nightfall is a technical measure to protect sensitive data and can prevent accidental exposure or access through real-time monitoring and remediation.

PIPEDA

In the context of the Personal Information Protection and Electronic Documents Act (PIPEDA) in Canada, there are several specific controls that relate to data leak prevention. Here are some of the controls and how they map to data leak prevention:

1. Safeguards Principle (Section 4.7): This principle requires organizations to protect personal information against unauthorized access, disclosure, copying, use, or modification. Implementing technical and organizational measures to prevent data leaks, such as encryption, access controls, and network security, helps ensure compliance with this principle. Nightfall is a technical measure that prevents data leaks and exposure through real-time monitoring and remediation.

2. Security Safeguards (Section 4.7.1): This control specifies that organizations must protect personal information using safeguards appropriate to the sensitivity of the information. It includes physical, technological, and organizational security measures to prevent data leaks, such as secure storage, firewalls, intrusion detection systems, and employee training. Nightfall can remediate sensitive data through different means whether that mean training an employee, redacting it, or deleting it, depending on the sensitivity of the information.

3. Access Controls (Section 4.7.3): Organizations need to establish controls to limit access to personal information on a need-to-know basis. By implementing user authentication mechanisms, role-based access controls, and strong password policies, organizations can prevent unauthorized access and data leaks. Nightfall monitors a SaaS environment in real-time so that sensitive data is not exposed in areas that it should not be in and thus contained on a need-to-know basis.

4. Training and Awareness (Section 4.7.4): This control emphasizes the importance of educating employees about privacy policies, procedures, and best practices to prevent data leaks. Training programs should cover topics like handling sensitive information, recognizing phishing attacks, and secure data disposal. Nightfall can notify end-users and employees that transmit sensitive data so that coaching is relevant and done in real-time to promote best practices.

5. Incident Response (Section 4.7.5): Organizations must have procedures in place to respond to and mitigate privacy breaches. This control includes developing an incident response plan, promptly investigating and addressing breaches, and notifying affected individuals and the appropriate authorities. Nightfall can scan a SaaS environment for sensitive data to determine if data was exposed.

6. Monitoring and Auditing (Section 4.7.6): Regular monitoring and auditing of security controls help identify and address vulnerabilities that could lead to data leaks. This control involves conducting security assessments, penetration testing, log analysis, and reviewing access logs to detect and prevent unauthorized access or data breaches. Nightfall can conduct a risk assessment to scan a SaaS environment and understand the nature of sensitive data exposure in the environment.

7. Disposal and Destruction (Section 4.7.8): Proper disposal of personal information is crucial to prevent unauthorized access and data leaks. Organizations should have policies and procedures for securely destroying or anonymizing personal data when it is no longer required. Nightfall remediation capabilities including actions like deleting and redacting sensitive data in real-time.

Healthcare

HIPAA

Data protection is intimated but not explicitly required for HIPAA compliance. It is intimated in the following ways:

• The Security Rule requires organizations to use appropriate administrative, physical and technical safeguards to ensure the confidentiality, integrity, and security of sensitive information.

  • Nightfall provides:

    • Administrative safeguards through highly contextualized end-user security awareness and training.

    • Technical safeguards by detecting and preventing clear-text personally identifiable information (PI) / protected health information (PHI) from leaving the environment, as well as automated corrective actions that lock down sharing or disallow download and print. Additionally, Nightfall supports the Audit Control standard by maintaining logs of PHI record creation or dissemination on the systems it monitors.

• Nightfall does not cover the other two rules, namely the Privacy Rule and the Breach Notification Rule. These rules are outside of Nightfall's domain as they respectively govern a patient's access to their records and the processes for managing breaches.

HITRUST

• Data Protection & Privacy:

  • Where possible and technologically feasible, the organization locates and removes/redacts specific PII or uses anonymization and de-identification techniques to allow the use of retained information while reducing its sensitivity and the risk from disclosure. Nightfall's automated remediation actions, such as deletion and redaction, support this control and serve as an overall risk mitigation for data disclosure.

  • The organization protects important records, such as contracts, personnel records, financial information, and client/customer information from loss, destruction, and falsification through the implementation of security controls. These controls include access controls, encryption, backups, electronic signatures, locked facilities or containers, among others. Nightfall's real-time monitoring enables organizations to detect and respond to sensitive data loss.

• Transmission Protection:

  • Before allowing the use of information systems for information exchange, the organization formally addresses multiple safeguards.

  • The organization never sends sensitive information without encryption using end-user messaging technologies (e.g. email, instant messaging, and chat). Nightfall can monitor SaaS applications that are not approved to confirm that PHI is not transmitted.

• Education, Training & Awareness:

  • Personnel are appropriately trained on leading principles and practices for all types of information exchange (oral, paper and electronic). Nightfall can assist teams in building enforceable information security policies and provide end-user security awareness and training that is highly contextualized.

Finance

PCI

PCI stands for Payment Card Industry, and it refers to a set of security standards and regulations designed to protect the sensitive cardholder data during payment card transactions. It ensures that businesses that handle payment card information maintain a secure environment to prevent data breaches and protect customer information. PCI is divided into 12 major sections, with several appendices, and 5 of 12 PCI controls are supported by Nightfall.

Version 3.2.1 and 4.0

Of the five sections, Appendix 3.2.6 is the section that most clearly states the requirement for DLP for all PCI-covered entities. As written:

• Appendix 3.2.6 - Mechanisms are implemented for detecting and preventing cleartext PAN from leaving the cardholder environment.

• Note that PCI’s example given explicitly states DLP: “Mechanisms to detect and prevent unauthorized loss of cleartext PAN [credit card number] may include the use of appropriate tools such as data loss prevention (DLP) solutions as well as manual processes and procedures.”

The following four PCI DSS sections are also relevant to DLP controls:

• Section 3: Protect Stored Account Data specifies that cardholder data must be protected while at rest. Additionally, the requirements state that methods for minimizing risk include not storing cardholder data unless absolutely necessary, truncating cardholder data if the full PAN is not needed, and not sending unprotected PANs using end-user messaging technologies such as email and instant messaging. Nightfall specifically covers these messaging technologies and can identify when such data is being stored somewhere where unauthorized access has occurred or may occur.

• Section 7. Restrict access to system components and cardholder data by business need to know: This point is the same as the one above. Nightfall can determine when need-to-know access might be violated.

• Section 10. Log and monitor all access to system components and cardholder data: While Nightfall is not an access control, it can help determine which users have had access to specific pieces of data within certain integrations.

• Section 12. Support information security with organizational policies and programs - Nightfall can help teams build enforceable information security policies. Additionally, it provides end-user security awareness and training that is highly contextualized.

Note on Version 4.0

No changes regarding DLP or additional requirements from prior version 3.2.1.

GLBA / FCRA

• Accounts numbers

• Credit card numbers

• Social Security numbers

These regulations are broad in nature and do not include specific enumerated requirements like PCI. However, they are explicit about not exposing customer records and impose fines for infractions (up to $100K per violation). Nightfall can be used as a compliance control to detect and prevent clear-text personally identifiable, payment, or account information from transmission. It also provides automated corrective actions that can lock down sharing or disallow downloading and printing.

SOX

Sarbanes-Oxley (SOX) is a financial reporting regulation that applies to publicly traded companies. It was created in response to the Enron collapse, which was caused by creative accounting of financial results. SOX focuses on the principles of integrity, accuracy, and completeness, and requires publicly traded companies to maintain confidentiality of their financial results before reporting.

Nightfall supports specific SOX requirements by detecting and preventing the movement of financial and accounting information in and out of the organization. It also provides automated corrective actions that restrict sharing and prevent downloading and printing.

• Section 302 of SOX requires the CEO and CFO to certify the accuracy of the financial statements. This certification requires a control to mitigate unauthorized access, alteration, or sharing of financial data.

Section 404 of SOX requires internal controls that focus on access, confidentiality of financial records, and their testing by management. Data leak prevention (DLP) solutions can be a part of those internal controls to protect sensitive financial data.

Tech Services

SOC 2

Version 2017

This version is the only currently accepted version of SOC 2 by AICPA.

For the SOC 2 sections specified, Nightfall supports these requirements through its monitoring, notification, and automated remediation:

• Confidentiality CC6.7: The entity restricts the transmission, movement, and removal of information to authorized internal and external users and processes, and protects it during transmission, movement, or removal to meet the entity’s objectives. This requirement specifically mentions data loss protection. Nightfall meets the standard to restrict the transmission and movement of data.

• Confidentiality CC6.1: The entity implements logical access security software, infrastructure, and architectures over protected information assets to protect them from security events to meet the entity's objectives. Nightfall protects the following SOC 2 specifically identified sensitive data:

  • Confidential information assets such as personal information.

  • Encryption keys during use, storage, and transmission.

• Confidentiality CC6.6: The entity implements logical access security measures to protect against threats from sources outside its system boundaries. Nightfall provides coverage of the cloud and protects against personal devices (BYOD) that may be outside the system boundaries, making it a Boundary Protection System. Nightfall also addresses a specific SOC 2 concern by protecting identification and authentication credentials outside of the system boundaries.

• Confidentiality CC2.2: The entity internally communicates information, including objectives and responsibilities for internal control, necessary to support the functioning of internal control. Nightfall's automated responses to end users provide real-time security awareness training that is highly contextualized to the specific violation identified.

ISO 27001

Version 2013

This version of ISO 27001 will no longer be valid after October 31, 2025. It includes limited data protection requirements that are detailed in a control supplement called ISO 27002. Two of these requirements can be supported with Nightfall:

• 11.2.6: Security of Equipment & Assets Off-Premises

  • Context: In the context of this section, data is considered the asset, and the cloud / SaaS is considered off-premises.

  • Security controls must be applied to off-site assets, which means protecting data (the asset) on the cloud (the off-site / third-party data center). ISO requires the same technical controls as if it were on-site. Nightfall supports the following ISO control sections:

    • 8.1.3 / 8.2.3: Acceptable use of information and systems - this refers to the appropriate handling (i.e., sharing / disclosure) of sensitive data, which is a prime directive of Nightfall.

    • 12.4.1 - 12.4.3: Monitoring of sensitive data use.

    • 16.1.2 - 16.1.3: Information Security Event Reporting.

    • 18.2.2 - 18.2.3: Compliance with policies and standards for information security.

• 13.2.1: Information Transfer Policies & Procedures

  • Communications must be secure regardless of the system used. Nightfall provides protection for communications over the public internet to various audiences, including Slack, Google Workspace, Jira, Confluence, and other systems. Thus, it is a control that supports the ISO requirement for confidentiality by taking into account the type, nature, amount, and sensitivity or classification of the information being transferred.

Version 2022

The latest version of ISO 27001 will be required after October 31, 2025. It includes the following new data protection requirements:

• A.8.12: Data leakage prevention. The requirement states that “data leakage prevention measures shall be applied to systems, networks, and any other devices that process, store or transmit sensitive information.” Therefore, it is recommended to use a tool like Nightfall when processing sensitive information, which is almost always the case.

• In support of the above requirement, Nightfall also serves as a control for:

  • A.8.10: Information deletion. Nightfall's automated deletion enables this requirement, which states that "information stored on information systems, devices, or in any other storage media shall be deleted when no longer required."

  • A.8.11: Data masking. Nightfall's data masking in protecting data is identified as a specific requirement. The requirement states that "data masking shall be used in accordance with the organization's topic-specific policy on access control and other related topic-specific policies, and business requirements, taking applicable legislation into consideration."

  • A.8.16: Monitoring activities. Nightfall is a monitoring control that fits within this requirement, which states that "networks, systems, and applications shall be monitored for anomalous behavior, and appropriate action taken to evaluate potential information security incidents."

• A.8.28: Secure coding. Nightfall's protection of secrets and keys, none of which should ever be disclosed in development, supports this ISO requirement. The requirement states that "secure coding principles shall be applied to software development."

Education

FERPA

The Family Educational Rights and Privacy Act (FERPA) directs educational organizations to keep student school records private and confidential. Nightfall can be used to detect and prevent the unauthorized transmission or storage of student education records, such as student names, addresses, grades, transcripts, and other personal information.

While FERPA does not specify control numbers, it does require educational institutions to:

• Ensure that administrative and technical controls are in place to protect student education records from unauthorized access, disclosure, alteration, or destruction. Nightfall is one such control as it protects personal data from unauthorized access, alteration, destruction, or accidental loss.

• Limit access to student education records to school officials who have a legitimate educational interest in the records. Nightfall can detect and prevent unauthorized access.

• Keep a record of all individuals who have requested or obtained access to student education records, along with the legitimate educational interest that each person has in obtaining the records. Nightfall can assist educational institutions in maintaining these records and detecting any unauthorized access to student education records.

Government

CMMC

The Cybersecurity Maturity Model Certification (CMMC) is a framework developed by the Department of Defense (DoD) to protect unclassified information. CMMC's scope is the Defense Industrial Base (DIB), which includes all organizations that support the DoD. This is defined as any organization that enables research and development, design, production, delivery, and maintenance of military weapons systems, subsystems, and components or parts to meet U.S. military requirements. In short, any system or component used by the DoD is covered by CMMC.

As of May 2023, version 2.0 is required. CMMC measures maturity and ranks it by levels.

Version 2.0

• 3.1.3: Control CUI Flow: Control the flow of CUI in accordance with approved authorizations. Nightfall supports this requirement by monitoring alerts and implementing automated remediations to maintain compliance with authorized CUI sharing on cloud systems.

• 3.8.5 Control access to media containing CUI and maintain accountability for media during transport outside of controlled areas. Nightfall supports protecting digital CUI. It applies mechanisms to prevent unauthorized loss or access to CUI via monitoring alerts and automated remediations such as access removal.

• 3.13.16 – Data at Rest: Protect the confidentiality of Controlled Unclassified Information (CUI) at rest. Nightfall’s automated remediation can help maintain confidentiality.

Note that Level 3 requirements have not yet been written. However, an additional requirement is currently under consideration.

• 3.1.3e - Employ secure information transfer solutions to control information flows between security domains on connected systems. This requirement calls for the use of secure information transfer solutions to control information flows between security domains on connected systems. Using Data Leak Prevention (DLP) to protect confidential data would satisfy this requirement. Therefore, Nightfall would help meet this requirement.

FedRAMP / NIST 800-53

NIST 800-53, which forms the basis for FedRAMP, is considered one of the less robust data protection security frameworks because it has fewer controls overall. Nightfall supports two major sections of the NIST CSF.

1. Information Flow:

   1. AC-4: Information Flow Enforcement - The information system enforces approved authorizations for controlling the flow of information within the system and between interconnected systems based on organization-defined information flow control policies. Nightfall enables this control through its policies and automated remediations.

2. Media Protection (note that the CSF definition of media includes laptops / mobile devices):

   1. MP-6 - Employs sanitization mechanisms with the strength and integrity commensurate with the security category or classification of the information. Nightfall's automated remediations, such as redaction, support this control.

   2. MP-7 - The organization restricts/prohibits the use of system media (in Nightfall's case, cloud systems/storage). Nightfall's automated sharing changes (e.g., Google Drive) support this control.

One of the first things you’ll do to get started with Nightfall is to choose which detectors you want to use. Ultimately, you will organize your chosen Detectors into in order to apply them to your cloud applications and infrastructure - but beforehand, we recommend that you carefully assess your needs and goals for sensitive data detection. Below, we suggest a few questions to ask yourself as you determine which detectors to leverage.

What data do you want to detect?

As discussed above, there is always a tradeoff between sensitivity and specificity. You want to plan your Detection Rules carefully to strike the right balance of accuracy without too much noise. The main question to ask yourself at this stage is what data you want to look for.

• Do you have a specific concern around a particular type of data (e.g. from a previous DLP issue)? In this case, you may be able to target a limited set of detectors.

• Do you want to implement a very broad DLP process, and don’t have a sense of which data might be problematic? Doing so will require more detectors, and thus means you may find a higher number of violations and/or noise.

Try to limit your detectors to data that you know or suspect are tracked or shared at your organization. For example, if you never discuss or store Vehicle Identification Numbers, then you likely do not need to set up a detector for that type of data.

How severe are the consequences of data leakage?

Some detectors will give more false positives due to their nature (e.g. widely occurring data types such as Dates, or less structured data types such as Names). In order to avoid excessive noise, you should weigh the relative risk of data exposure. For example, it may be problematic for your organization to leak Credit Card Numbers, but not Dates.

How much does the data’s context matter?

Another way to eliminate noise is to create specific Detection Rules that scan for combinations of data types. For example, it may be problematic for your organization to have Dates appearing alongside Names, but not for Dates to appear alone.

Nightfall allows you to create custom detectors that match your organization's requirements. If none of the existing Nightfall detectors match your requirements, you can use this option to create a custom detector.

Nightfall provides you with four types of custom detectors.

• Dictionary: This Detector allows you to upload a text file. The text file must have a specific token which is sensitive data for your organization. The Detector looks for this token in all the data and if a match is found, it flags it as sensitive data found.

• File Type: This Detector allows you to select a (Multipurpose Internet mail extension). (roughly speaking, every file on the Internet is recognized by its format which is called MIME). The Detector scans all your data to check if it has a file with the selected MIME type.

• File Fingerprint: This Detector accepts a file as input. You can add all the content that you feel is sensitive to your organization. The detector uses this file as the base and scans your data to check if any matching content is present. If found, it flags it as sensitive data.

• Regular Expression: This Detector allows you to define regular expressions matching the . The Detector scans all your data to find any content that matches the regular expression. If found, it flags it as sensitive data.

• Extend a Nightfall Detector: This Detector type allows you to customize an out-of-the-box Detector by defining Context Rules and Token rules.

To learn more about how to create each of the custom detectors, you can refer to the following documents.

This document lists all the out-of-the-box detectors provided by Nightfall.

Common Entity Detectors

Standard PII - AMERICAS

Standard PII - EMEA / AP

Standard PII - APAC

Finance - Banking & PCI

Health

Hardware and Network

Secrets

Image Documents

The Health Insurance Portability and Accountability Act of 1996 (HIPAA) is a federal law protecting the disclosure of individuals' health information (aka protected health information or PHI).

Nightfall's PHI detector identifies PHI with high accuracy and relevancy using 15 PII and healthcare entity detectors combined, as described in Tables 1 & 2.

Table 1: Nightfall PHI detector = Nightfall PII + Healthcare detectors in specfic combinations

Please refer to the for more information on the individual detectors above.

Table 2: Nightfall PHI detection confidence

A dictionary detector can detect a specific piece of sensitive data (referred to as a token). The token can be any type of sensitive data like a password, API key, IP address, PII, and so on. You must add the token to a file and then upload the file while creating the dictionary. Nightfall scans your data to check if the token present in the uploaded file exists in your data. If a match is found, it fis flagged as sensitive data exposure.

To create a Dictionary Detector:

1. Navigate to the Detectors section from the left pane.

2. Click + Custom Detector and select Dictionary.

1. Enter a name for your custom Detector in the Name field.

2. (Optional) Enter a description for the Detector in the Description field.

3. Upload or drag and drop the text file which contains a token.

4. (Optional) Select the Match substrings check box to match the sub-strings of the text file.

5. Click Add.

Welcome to Nightfall Documentation. A one-stop shop for all your queries about the Nightfall DLP product.

In order to evaluate the success of your cloud DLP detection, you must first have clarity around your organization’s goals. This will be highly dependent on your organization’s unique circumstances and needs.

Here are some guiding questions:

• What sensitive data would you like to detect and which are the highest priorities for each SaaS application?

• What level of risk are you comfortable with? Would you like to alert on only the most risky findings or have visibility into even low-risk data types that may not warrant action?

• How much operational load are you willing to take on? The more granular your approach, the less human discretion will be required in the filtering and remediation steps.

• What are the requirements of any compliance regimes you are accountable to?

• What user strategy would you like to implement?

  • Some potential answers:

    • Minimal impact on user experience, only alerting and monitoring to security staff and notifications to end users to guide towards remediation?

    • Educating and empowering users about DLP best practices?

    • Enforcing DLP through strict controls and automation?

Testing Detection Rules

Once you have a sense of your goals, we recommend that you optimize your detection during a phased, iterative implementation - start small, and broaden gradually. Typically, the biggest pitfall customers want to avoid during implementation is to get flooded by too many low-priority alerts that become unmanageable - which is typically caused by taking too broad of an approach to detection. By taking the time to optimize your detection gradually during the first few weeks, your team will be set up for ongoing success.

For this reason, we have created our Detection Wizard, which is a resource that will recommend the exact configuration to suit your use cases. You can shortcut a lot of legwork in choosing and fine-tuning rules by immediately identifying the recommended and validated configuration that matches your use case.

There is generally little reason to deviate from the Wizard’s recommendations at the outset, but your use case may warrant fine-tuning and broadening your detection logic over time:

We recommend that you select just a few detectors to start out with as recommended by the Wizard, in order to optimize detection of the most critical and high-risk information types.

• Once a few detectors and initial detection rules are in place, test some sample data and monitor the results. We suggest you evaluate metrics such as false positive and false negative rates, and assess whether the volume of alerts/results is manageable by your team. You may adjust parameters such as Minimum Confidence and Minimum Number of Findings to optimize the level of alerts through your testing process.

• Then, continue to iterate and modify your detection by adding and optimizing a few more detectors as required. Focus on starting with tighter detection rules (higher confidence levels), and slowly relax or broaden the rules over time.

Sample Data

Nightfall offers a library of Sample Data that we recommend for evaluating DLP technologies.

Evaluating Detection During Tests vs. in the Wild

It’s important to understand that the rate of true positive data “in the wild” tends to be significantly lower than true positive data in a test environment. That’s because, in a test environment, many occurrences of sensitive information will be intentionally planted.

As a result, test environments are typically most useful for optimizing sensitivity, or the true positive rate - detection is optimized toward identifying sensitive information as sensitive. While this helps ensure that occurrences of sensitive information are, in fact, correctly flagged, it also means detection is somewhat skewed toward flagging information as sensitive, thus increasing the potential for false positives (see the tradeoff between sensitivity and specificity).

What this means for you is that if you fine-tune a detection rule in a test environment, you may over-index towards sensitivity rather than specificity, which can result in a higher degree of false positives in the wild. As a result, we highly recommend evaluating detection rules in a production environment rather than a test environment, when possible.

Background Knowledge

An Integration in Nightfall refers to a SaaS application in which your organization's data resides. This SaaS application can be Google Drive, JIRA, Notion, GitHub, and so on. The sensitive data residing in these applications is highly prone to data leakage, accidentally by your employees.

You can connect your SaaS applications to Nightfall by adding them as integrations. Once a SaaS application is added as an integration, Nightfall continuously scans it to check if there is any sensitive data leakage.

Understanding Policies

A Policy in Nightfall allows you to define a specific part of your integration or the entire integration to be monitored for sensitive data leakage, by Nightfall. Apart from monitoring your environment, Policies in Nightfall also allow you to configure alerts and automatic actions if Nightfall discovers sensitive information in the integration being scanned.

Once you create a policy, if there is any instance of data leakage, Nightfall sends notifications to the audience configured by you in the policy. It also takes automated actions, configured by you in the policy. If you have disabled automated actions and no action is taken by you, Nightfall also sends reminders, as per the frequency set by you in the policy. All these actions are done automatically by Nightfall, as per the policy settings, configured by you.

A Policy in Nightfall is always associated with integration. You can create multiple policies on a single integration. Before creating a policy for an integration, you must first ensure that the integration is added to Nightfall.

Policy Stages

A Nightfall Policy has multiple stages. You can configure various settings like adding specific sections of integration to be scanned, configuring alert notifications, end-user actions, and so on. The various stages of a policy are described in the following document.

Selecting Integration

Scope of the Policy

Detection Rules

Advanced Settings

Name and Risk Score

Some Nightfall integrations (e.g. Nightfall DLP for Google Drive) offer options for both historical and real-time scanning. Some organizations wish to minimize the amount of real-time alerts they receive daily but still want to maintain a comprehensive data scanning and protection strategy.

One option is to create Detection Rules for real-time scans that are limited to the types of sensitive information that are most critical or highest risk to your organization. Then, leverage broader historical scans to capture less critical risks on a routine cadence, when your team can plan resourcing in advance (e.g. monthly or quarterly).

Running Historical Scans

Nightfall can fully manage historical scan executions on your behalf to scan data in your SaaS applications. Due to the sheer volume of content that can amass in cloud apps over time, historical scans can be very resource-intensive and may take days or more to process. As such, we advise against running broad historical scans all at once.

Instead, we recommend that you prioritize certain detectors based on your organization’s definition of critical violations and limit the scan to a specific date range (e.g. one month at a time). Then, you can request the Nightfall team (support@nightfall.ai or contact your customer success representative) to run a historical scan on a SaaS application of interest.

Upon completion of the historical scan, Nightfall will share a risk assessment report with an overview of the highest severity findings, highest risk files or resources, users with the most number of violations, the total amount of data scanned, and more.

In many cases, you may not need all of your data, residing in the integration, to be scanned. You might only require a specific section of your data to be scanned which is highly prone to data leakage.

The Scope stage allows you to set boundaries for monitoring. You can pick only a required section of the integration to be scanned thus reducing the noise from trusted sources. All the integrations in Nightfall (except Notion and ChatGPT) provide you the flexibility to pick and choose specific sections to include or exclude for monitoring. Nightfall scans only the data that matches the scope settings configured by you.

The configurations of the Scope stage varies for each integration. The following sections briefly describe the Scope stages for all the integrations.

GitHub

The GitHub integration's Scope stage allows you to select a specific GitHub org initially. Once you select the org, you can choose to scan either all of the repositories, only the public repositories, or only the private repositories from within the selected GitHub org. Once you select the required repository type (all, public, or private) to be scanned, Nightfall allows you to set conditions to exclude a specific repository (or repositories) which you do not wish to scan.

Microsoft Teams

The Scope stage in the Microsoft Teams integration allows you to set the scope on two entities.

a. Chats: The Chats section allows you to scan individual chat messages sent in MS teams. You can choose to scan either messages from all the users or specific users and groups. Furthermore, if you choose to scan messages from all the users, Nightfall optionally allows you to exclude scanning of specific users and user groups.

b. Teams: The teams section allows you to scan messages shared in Teams. You can choose to scan either specific Teams or specific Teams. Within the selected team, you can choose to scan the required channels. You can also use exclusion rules to exclude specific channels or Teams.

Slack

The Scope stage in the Slack integration is different for Slack Pro and Business+ editions, and Slack enterprise edition.

a. Slack Pro and Business+: In Slack pro and business+ editions, you can choose to monitor all the public channel, private channels, public connect channels, or private connect channels. You can choose to scan the required channel type(s). Within the selected channel type(s), Nightfall allows you to scan or exclude scanning of individual users, groups, or apps.

b. Slack Enterprise: In Slack Enterprise edition, you can choose to scan either specific workspaces or specific channels. If you choose to monitor Workspaces, you can further choose to monitor all the types of channels and connect channels. Furthermore, you can choose to scan or exclude scanning of specific Users, groups, channels, or apps.

JIRA

The Scope stage in JIRA integration allows you to select a specific JIRA instance initially. Once you select a JIRA instance, you can choose to scan either all the projects or specific projects from the selected JIRA instance. If you choose to scan all the JIRA projects, Nightfall allows you to exclude trusted JIRA projects from being scanned.

Confluence

The Scope stage in Confluence integration allows you to select a specific Confluence site initially. Once you select a Confluence site, you can choose to scan either all the Spaces or specific Spaces from the selected Confluence site. If you choose to scan all the Confluence Spaces, Nightfall allows you to exclude trusted Confluence Spaces from being scanned.

Google Drive

The Google Drive Scope stage allows you to select all User Drives or shared drives. Within shared drives you can choose to scan either specific shared drives, exclude scanning of specific shared drives, or scan all the shared drives. Furthermore, you can also choose to scan files based on their sharing permissions. Lastly, Nightfall provides a host of filters to for granular controls. You can use the filters, to include or exclude specific users, groups, files, or labels.

Gmail

The Gmail Scope stage allows you to scan or exclude the scanning of emails sent by specific users or user groups. Apart from senders you can also choose to scan or exclude scanning of emails sent to specific recipients and domains.

OneDrive

The Scope stage in OneDrive allows you to select files either based on permissions or special folders. Once you select the files based on permissions and special folders, you can choose to exclude certain files by creating specific exclusion rules.

Zendesk

The Zendesk Scope stage allows you to select a Zendesk instance. Once you select a Zendesk instance, you can choose to scan messages based on status of the tickets. Once you choose tickets with the required status, you can choose to exclude tickets from specific groups or agents.

Salesforce

The Salesforce Scope stage allows you to select a Salesforce org initially. Once you select a Salesforce org, you can choose to scan the required objects within the org, and the fields within the selected objects.

Best Practises to Create Policy Scope

Policy scopes allow you to configure which data to scan in your connected SaaS, GenAI applications. By carefully defining your policy scopes, you can reduce alert volume and focus on the most critical data. Here are some key considerations and best practices for creating effective policy scopes:

General Considerations

• Start broad, then narrow: Begin with a wide scope and gradually refine it based on your findings and needs. You can execute a historical scan on supported SaaS apps to assess where and what types of sensitive data reside in these apps and use this information to create policies.

• Prioritize sensitive data: Focus on areas where sensitive or regulated data is most likely to be stored or shared.

• Consider your organizational structure: Align scopes with your company's departments, teams, or projects.

• Review and update regularly: As your organization changes and as you start reviewing policy violations from Nightfall, revisit and adjust your policy scopes accordingly.

App-Specific Considerations

Slack, Microsoft 365 Teams

• Create scopes based on channel types such as public, private or connect channels or DMs.

• Utilize users, user groups to limit scanning of messages sent by specific users or users within specific groups.

Google Drive

• Use file permissions to limit the scanning of files that are externally shared or shared with external users, groups

• Leverage users, user groups synced via Okta/Google Directory/Entra ID for team or department-wise scoping. Implement label-based filtering for sensitive document categories.

Gmail

• Focus on external communications by filtering specific recipient domains or emails sent by specific senders, and recipients.

• Create separate policies for different departments or teams based on email patterns. For example, you may only want to monitor email communications of the support team to specific recipient domains.

Salesforce

• Focus on specific objects and fields containing sensitive customer data.

OneDrive

• Utilize file permissions to focus on shared or externally accessible content.

• Consider scoping by specific user drives for high-priority individuals.

• Use file type exclusions to ignore non-sensitive formats.

• Leverage label-based exclusions for efficient filtering.

Jira, Confluence

• Scope by including or excluding specific projects in Jira.

• Use space inclusions/exclusions to focus on specific areas of your knowledge base.

• Consider separate policies for public and restricted spaces.

GitHub

• Scope by each organization within GitHub

• Use repository pattern exclusions to ignore non-sensitive code bases.

• Implement file extension filtering to focus on specific file types.

• Utilize path exclusion to ignore non-critical areas of repositories.

Zendesk

• Create scopes based on ticket status to prioritize discovery and remediation of sensitive data within closed issues.

Best Practices for Implementation

• Test policies with limited scopes before expanding.

• Monitor alert volumes and adjust scopes as needed.

• Collaborate with stakeholders from different departments to ensure comprehensive coverage.

• Regularly review and update scopes as your SaaS usage evolves.

By following these best practices and considering the unique filtering capabilities of each supported app, you can create effective policy scopes that balance comprehensive data loss prevention with manageable alert volumes.

Yes! While Nightfall's pre-built detectors listed above are trained via machine learning, Nightfall also supports RE2 regexes and word lists for any custom detectors that may be of interest to you. Over time, we've aggregated the following Regex Library, which you're welcome to select from to save you some time.

Please note that a regular expression is an established yet limited method that searches for pre-defined patterns, so your mileage may vary.

You can test regular expressions here.

You can input custom detectors directly in the Nightfall console at app.nightfall.ai by navigating to Detectors → New Detector → Regular expression.

Nightfall supports scanning most file types, except for audio and video files. Image files are scanned using optical character recognition (OCR). Embedded files images and non-visible URLs are included in file scans. Max file size is 1 GB. Contact us at support@nightfall.ai, if you need us to raise this limit.

Supported file types:

• Microsoft Office documents (Word, Excel, Powerpoint, etc.)

• Google Workspace documents (Doc, Sheet, Slide, etc.)

• Image file types (PNG, JPG, TIFF, GIF, etc.)

• PDF files

• All text-based document files (HTML, TXT, etc.)

• All text-based code and config file types

• All text-based data file types (CSV, TSV, JSON, XML, parquet, etc.)

• Compressed file types (zip, gzip, etc.) are decompressed, and the contained files are scanned.

Unsupported file types (audio, video, and animation):

• audio/midi

• audio/mpeg

• audio/wav

• audio/x-midi

• video/mp4

• video/mpeg

• application/photoshop

• video/Quicktime

• Autodesk 3d animation files (ma, fbx)

Social security numbers (SSNs) are a common data type we scan for. You may notice 9-digit numbers in your data that resemble SSNs. If you lower your detection rule confidence to possible, these numbers will appear in your dashboard.

However, over 90% of 9-digit numbers are typically false positives that don't require action. Numbers formatted like SSNs are often used as internal IDs - for tickets, service calls, events, dispatches, transactions, etc. They can also be valid driver's licenses or bank account numbers.

With more context, our models can better determine if a 9-digit number is likely an SSN. For example, when a number appears in a sentence or tabular data with a descriptive header, our confidence in predicting it as an SSN increases to likely or very likely.

Providing contextual cues helps our models accurately identify SSNs and avoid false positives. We continue improving our algorithms to balance detection with precision.

The same thinking applies to other alpha-numeric info types like credit card numbers and passport numbers.

Please reach out if you have any further questions!

This stage is optional. This stage allows you to configure notification alerts, automated actions, and end-user actions. This stage has three components. Before we proceed further, you need to understand the following terms.

• Admin: These are the users in your organization who work on Nightfall. These users configure integrations, create Policies, monitor Dashboards and Violations, and perform other actions in the Nightfall console. These users can be called Nightfall administrators.

• End-User: These are the users who do not work on the Nightfall console but work on other SaaS applications in your organization. These users can be the members of your software development teams, QA engineers, network administrators, and so on. End-users are more prone to leaking sensitive data from your organization.

The Advanced Settings stage has the following components.

Admin Alerting

In this stage, you can configure notifications. The Nightfall admin receives the notifications configured in this stage. The various notification channels available are as follows.

• Slack Alerts: You can configure Slack as a notification channel. When you configure Slack as an alert channel, admin users get a notification each time the policy has been violated. To learn more about configuring Slack as an alert channel in policies, see this document.

• JIRA alerts: You can configure JIRA as an alert platform for policies. When the policy is violated a new JIRA ticket is created in the JIRA project of the JIRA instance selected by you and the Nightfall admin receives a notification in JIRA for the creation of the new ticket. You can refer to the Setting Up JIRA as an Alert Platform to learn more about using the alert platform.

• Email Alerts: You can configure Email notifications and enter the Email IDs of the Nightfall admin users. The admin users receive an email when the policy is violated.

• Webhook Alerts: You can configure Webhook as an alert platform. You can refer to this document to learn more about webhook configuration.

Automated Actions

The Nightfall automated actions are neither performed by the Nightfall admin nor the end-users. They are triggered automatically when a violation is detected. Nightfall admin only needs to enable the automated actions. Automated actions are not present in all the integrations. Nightfall admins can either choose to trigger automated actions immediately when a Nightfall is detected or after some time.

The various automated actions are described as follows.

• Redact: This action redacts the sensitive data that caused the violation. Admins can choose to implement this action either immediately when the violation is detected or after some time.

• Delete Attachment: This action deletes the attachment that contains sensitive data. Admins can choose to implement this action either immediately when the violation is detected or after some time.

• Remove Access: This action revokes public access to the page that contains sensitive information and makes it private or revokes page access to guest users (external users who are not part of your organization. Admins can choose to implement this action either immediately when the violation is detected or after some time.

End-User Notification

The settings configured by the Nightfall admin in this stage send notifications to end-users (users whose actions triggered violation). Furthermore, end-users can also take remediation actions (human firewall) from within the notifications, provided the Nightfall admin configures the required settings.

This stage has the following configurations.

• Custom Message: In this section, you can draft a custom 1000-character violation message to be sent to end-users. By default, Nightfall provides you with a violation message. You can use it or draft your custom message.

• Automation: This section allows you to choose the notification channel through which the above message must be sent. Slack and email are some of the supported channels. The channels vary for each integration.

• End-user Remediation (Human Firewall): In this section, Nightfall admins can configure remedial actions that end-users can take when they trigger a violation. The available actions vary for each integration. If end-users do not perform any of the remediation actions after triggering a violation, Nightfall admins can set reminders and also the frequency at which the reminders must be sent.

Nightfall's API Key Detector

Nightfall's API key supports generic API key detection plus vendor-specific and use-case-specific keys. Nightfall attempts to validate the key status of vendor-specific keys and JWT tokens.

Nightfall's Cryptographic Key Detector

Nightfall identifies and validates popular crypto keys for locking or unlocking cryptographic functions, including authentication, authorization, and encryption.

You can send us a request for new Secrets detectors directly in Nightfall.

A File Type detector allows you to select MIME (multi-purpose mail extension) types. The selected MIME types are considered as tokens and Nightfall scans your data to identify if there is anything that matches the MIME token. If a match is found, it is flagged as sensitive data.

To create a File Type Detector:

1. Navigate to the Detectors section from the left pane.

2. Click + Custom Detector and select File Type.

3. Enter a name for your custom Detector in the Name field.

4. (Optional) Enter a description for the Detector in the Description field.

5. Click Add File Type and select an MIME type.

6. (Optional) Repeat step 3 to add multiple MIME types to your rule.

7. (Optional) Click the delete icon to delete an MIME type, if you do not require it.

8. Click Add.

As described in the Nightfall Detection Engine once you have finalised the the Detectors to be used or created custom detectors which leverage your organization's sensitive data security requirements, you must add the detectors to a detection rule and then add the detection rule to a policy. You cannot use detectors directly. Detectors become useful only when you add them to a Detection rule and then add the detection rule to a policy.

You can add multiple detectors to a single detection rule. Nightfall does not restrict you on the type of detector that can be added to a detection rule. You can add both, a custom detector and a Nightfall detector to the same detection rule.

You can create multiple detection rules to detect different types of sensitive data. For example, you can add the Password detector to a Detection rule and use it to detect password leaks. You can add the API key detector to another detection rule and use it for detecting API key leaks.

Once you add all the required detectors to the detection rule, Nightfall allows you to test the detection rule in the Nightfall Playground. You can use the Nightfall playground only for testing if the detection rule works as expected. Once you confirm that the detection rule works as expected, you must add the detection rule to policy to leverage your data security requirements. Conversely, you can also have a single detection rule in which you add both Password and API key detectors.

You can access the Detection rules page by clicking Detection from the left pane and then clicking Detection Rules. To create a new Detection rule, you must click the + New Detection Rule button the top right corner.

Adding Detectors to Detection Rule

You can add a Detector to the Detection rule by executing the following steps.

1. Click the + New Detection Rule button the top right corner.

2. Enter a name for the detection rule in the Name field.

3. (Optional) Enter a description for the detection rule in the Description field.

4. Click + Add Detectors to add detectors to the detection rule.

1. Select the check box for the detectors that you wish to add to the detection rule. You can use the filters to view only the Nightfall detectors or the custom detectors or the search bar to search detectors.

2. Click Add.

You can see that the selected detectors are added to the detection rule. In the following image two detectors have been added to the Detection rule.

Detection Rule Settings

Once you add the required detectors to the detection rules, you can se that there are three columns; Scope, Minimum Confidence, and Minimum # of Findings. These settings define the weightage of the detector in the detection rule. You must configure these settings for each detector.

These settings are explained in the following sub-sections.

Scope

The Scope setting allows you to define which part of the data the detector must scan. You can configure this setting to scan either the name of the file, the contents of the file or both.

If you select the Content option, this detector only scans the content of the files and not the name of the file, and vice versa, if you select File Name. However, if you select Both, the detector scans the file name as well the contents of the file.

For example, if the API key detector has to scan a Google Drive file and if you select, File Name, then the detector only scans the name of the file to check if the name has any API key. Similarly, if you select Content, then the detector only scans the content of the file and not the name. If you select Both, then the detector scans both; the name and the contents of the file.

Minimum Confidence

The minimum confidence setting defines the probability of a detector's findings being actual violations. Nightfall provides you with three Minimum Confidence settings; Possible, Likely, and Very Likely.

The Possible option indicates that there is a 40-60% chance of the Detector's finding being an actual case of data leak, Similarly, the Likely option indicates that there is a 61-80% chance of the Detector's finding being an actual case of data leak. The Very Likely option indicates that there is a 81-100% chance of the Detector's finding being an actual case of data leak.

You can set the Minimum Confidence setting for each detector. When the detector detects any data leak, it is logged in the Nightfall Events page with the Minimum Confidence setting configured here.

Minimum Number of Findings

This setting defines the minimum number of data leak cases that a detector must encounter for it to be considered as a Violation by the Detection rule. You must configure this setting for each detector. For example, if you set this setting to 10, the detection rule does not consider the first 9 occurences of data leak encountered by the detector. Detection engine starts considering the 10th and all subsequent cases of data leak to be actual case of violations.

Applying Logical Operators Between Detectors

If a detection rule has a single detector and if that detector detects a data leak, the detection rule is triggered and it is logged as a Finding.

However, when you have multiple detectors in detection rule, you can choose to trigger the detection rule when any of the detector detects a data leak or when all the detectors detect a data leak.

As you can see in the above image, you can choose the flag a Finding of the detection rule either when any single detector detects a data leak or when all the detectors of the detection rule detect a data leak.

Understanding When the Detection Rule Triggers

The above image has two detectors. The following statements hold good.

• The Scope of both the detectors is set to scan the file content and file name.

• When any of these detectors detect a data leak, it is logged as a Finding in the Violations page and tagged as Very Likely.

• The minimum number of findings for each detector is 1. So even if there is a single case of a data leak, a detector rule considers it as a finding.

• The detection rule is triggered even if a single detector detects a data leak.

Testing Detection Rules in Nightfall Playground

Once you create the detection rule, it is highly recommended that you test the rule to ensure that it returns the expected results, when added to policies. Nightfall provides you with a Playground to test the detection rules.

Prerequisites to use Nightfall Playground

• You must have an API key to use the Nightfall Playground. You can refer to this document to learn about how to create an API key.

• You must have some sample data to test the Detection rule. This Nightfall document has various types of sample data sets that you can use.

Capturing Detection Rule UUID

Each Detection rule in Nightfall has a UUID. This is a unique ID assigned to each detector when it is created. You cannot edit the UUID of any detector. You must capture this UUID to use it in Playground to test the detection rule.

To capture Detection rule UUID:

1. Navigate to Detection rules.

2. Click the detection rule whose UUID has to be captured.

3. Click the UUID to copy it.

Using the Nightfall Playground

To use the Nightfall playground:

1. Navigate to the Nightfall playground.

2. Use either the I'm Scanning Text or I'm Scanning Files tab based on whether you wish to scan text or file.

What is machine learning?

Each machine learning detector is a model, which is trained using curated datasets to recognize sensitive data tokens based on certain characteristics (e.g. structure) and on surrounding context (e.g. preceding words). Machine learning models have the ability to automatically learn and improve from experience, without being explicitly programmed - giving them broader scope and higher accuracy than hard-programmed models (such as regular expressions).

How do Nightfall’s detectors leverage machine learning?

Nightfall’s detectors are trained via machine learning to recognize many potential permutations of sensitive data tokens, and also to recognize and assess the surrounding context. The ability to recognize a variety of data structures and context gives Nightfall’s detection far more scope and accuracy than traditional regular expression (regex)-based detection. Nightfall’s internal team of machine learning engineers and data scientists are continuously developing new machine learning-based detectors, and they also handle the ongoing maintenance and training of our detection engine to continually improve accuracy and results.

Understanding Accuracy

A little background on measuring accuracy will be helpful to understand before you embark on your detector optimization journey. Accuracy is the ability of a detector to correctly flag sensitive content and not flag content that is not sensitive.

No detector is 100% accurate, and it is possible to have a discrepancy between information that is actually sensitive, and information that is flagged by a detector as sensitive. For example, a detector may incorrectly assess a piece of sensitive content as nonsensitive (this would be a false positive). The various result types you may get, based on the actual versus assessed sensitivity of content, are illustrated below.

Accuracy = (TP + TN) / (TP + TN + FP + FN)

Sensitivity = TP / (TP + FN)

Specificity = TN / (FP + TN)

Your organization will, of course, want sensitive data detection to be as accurate as possible. However, it’s important to understand that there is always a tradeoff between accurately identifying true positives (sensitivity) versus accurately identifying true negatives (specificity).

• Sensitivity (also known as: recall, hit rate, or true positive rate [TPR]) is the ability of a detector to correctly identify content with sensitive findings.

• Specificity (also known as: selectivity, true negative rate [TNR]) is the ability of a detector to correctly identify content without sensitive data.

The tradeoff between sensitivity and specificity occurs because you must choose whether to optimize a detector toward or away from flagging information. Higher sensitivities mean lower specificities and vice versa. For example, to optimize for sensitivity, you may enable more detectors at lower minimum confidences to catch more true positives. In doing so, you will open yourself up to more false positives, which will lower your specificity.

As a result, each organization should spend time optimizing its detectors to strike the right balance between sensitivity and specificity.

Confidence Levels

A confidence level represents Nightfall’s confidence in the accuracy of a given detection. Each detector is a unique model and will have its own confidence level profile. For details on how confidence level is determined for a given detector, please reach out to us.

Confidence may be any of the following values corresponding to the given confidence intervals:

Each info type is unique, so it is not possible to dictate that what yields a “Very Likely” match for one detector, will yield a “Very Likely” result for another. Nonetheless, we can provide some high-level guidance. Generally speaking, higher confidence detections may have certain features that increase confidence in the detection being accurate - some examples of the features include:

• Formatting of token

• Passes validation functions

• Passes substring checks

• Context clues that are indicative of that info type

• Model scoring

In this stage, you select an integration for which the policy is being created.

To select an integration:

1. Click Policies from the left menu.

1. Click + New Policy.

1. Select Sensitive Data.

1. Select the integration for which you wish to create the policy.

Based on the integration selected, the options vary in the upcoming policy stages.

A File Fingerprint detector allows you to upload a file. You must enter the exact data that is sensitive to your organization, in this file. Nightfall scans your data to check if there is an exact match between the contents of the uploaded file and your organization's data. If a match is found, it is flagged as sensitive data.

To create a File Fingerprint Detector:

1. Navigate to the Detectors section from the left pane.

2. Click + Custom Detector and select File Fingerprint.

3. Enter a name for your custom Detector in the Name field.

4. (Optional) Enter a description for the Detector in the Description field.

5. Upload or drag and drop the text file whose content needs to be matched.

6. Click Add.

You can customize a Nightfall detector to meet your organization's requirements. Once you customize a Nightfall detector, it is considered to be a custom detector. The Nigytfall detector that you used and customized, continues to exist as is.

Nightfall provides you with two customization options. Context rules and Exclusion rules. You must use at least one of these two rules to customize a Nightfall detector. The procedure to use Context rules and Exclusion rules is the same as in the case of Regular Expression Detector. You can learn how to configure Context rules and Exclusion rules from the#understanding-context-rules-and-exclusion-rules document.

To extend a Nightfall Detector:

1. Navigate to the Detectors section from the left pane.

2. Click + Custom Detector and select Extend a Nightfall detector.

3. Select the Nightfall detector that you wish to customize.

4. Enter a name for the customized detector in the Name field.

5. (Optional) Enter a description for the detector in the Description field.

6. Click Next.

1. Click + Add Context Rule to configure the context rule.

2. Click Save to save the rule.

3. Click Next.

4. Click + Add Token Rule to configure token rules.

5. Click Save to save the rule.

6. Click Next.

1. Review the configurations and click Add.

A regular expression detector allows you to define a regular expression pattern. Once you define a pattern, Nightfall scans your data to check if there is anything that matches the given pattern. If a match is found, it is flagged as sensitive data. You can also refer to this link to generate regular expressions.

Understanding Context Rules and Exclusion Rules

To use the Regular Expression detector effectively, Nightfall provides you with two special rules; Context Rules and Exclusion Rules. When Nightfall scans your data and finds some sensitive data, these rules are used to further scrutinize the sensitive data to be sure that the data is sensitive. Context rules and Exclusion rules are optional and you can define them only if you wish to have added filtration on sensitive data to be sure that data that is termed as sensitive, is sensitive.

Context rules are a set of hot and cold words that help Nightfall to effectively identify sensitive data.

Exclusion rules define specific scenarios in which data flagged by a detector should be excluded from the list of findings.

You can find these rules in the Regular Expression detector. You must click + Rule to create a Context rule or Exclusion rule.

Context Rules

Context rules help Nightfall identify sensitive data by providing data about the text surrounding sensitive data. You can define a regular expression pattern to match the data that usually surrounds sensitive data. You can then instruct Nightfall as to where this surrounding data can be generally found. It can be found before sensitive data, after sensitive data, or both before and after sensitive data. You can also define confidence level settings when a match is found.

For example, credit card data is sensitive data. If a user shares it accidentally online, then they generally draft the following phrases before disclosing the credit card number.

my credit card details are

my credit card number is

Please find below my credit card details

My credit card number is as follows

After disclosing the credit card information, users generally draft the following messages.

Please find my credit card details above

I have shared my credit card details

Shared my credit card details. Please do the needful

As requested, shared my credit card details. Let me know if you need any further 
details

Context rules involve creating regular expressions to define these types of phrases that surround the sensitive data. You can use this link to generate regular expressions to define data that surrounds sensitive data. You can define the Context rule regular expression in the Pattern to match text box.

You can also choose to make your regular expression case-sensitive by selecting the Case sensitive check box.

Windows Setting: Once you create the regular expression to define the possible phrases that can surround the sensitive data, you can use the Windows setting to instruct Nightfall as to where exactly this surrounding text can exist around the sensitive data (before, after). Nightfall provides you the following Windows settings.

• Before the finding: You can use this setting to define the number of characters before the prospective sensitive data at which surrounding text can be found. For example, if you define this setting as 20 characters before the finding, and if Nightfall detects the prospective sensitive data on line 4 column 60, it looks at the data on line 4 and from column 40 to check if it matches the defined regular expression for Context rules.

• After the finding: You can use this setting to define the number of characters after the prospective sensitive data at which surrounding text can be found. For example, if you define this setting as 20 characters after the finding, and if Nightfall detects the prospective sensitive data on line 4 column 60, it looks at the data on line 4 and from column 60 to check it that matches the defined regular expression for Context rules.

• Before or after the finding: You can use this setting to define the number of characters before or after the prospective sensitive data at which surrounding text can be found. For example, if you define this setting as 20 characters before or after the finding, and if Nightfall detects the prospective sensitive data on line 4 column 60, it looks at the data on line 4 and from column 60 and also on data at line 4 from column 40 to check it any of the two match the defined regular expression for Context rules.

Confidence Change: If Nightfall finds any data to be sensitive and if this prospective sensitive data matches the Context rules as well, you can then define Confidence change settings to instruct Nightfall on what to label this prospective sensitive data. Nightfall provides you with four Confidence change settings

• Exclude: If you set the Confidence Change setting to Exclude, Nightfall does not consider the prospective Finding to be actually sensitive (even though it matched both; sensitive data's regular expression and also the Context rules' regular expression and Windows setting) and excludes it from Findings. You can use this setting to eliminate false positive findings. For example, consider that one of your prospective customer is testing your API and you need to share an API key for this testing. You share a dummy API key and do not wish this API key to be flagged as sensitive data. You can ask your employees to draft a specific phrase before sharing the API key (something like, the dummy API key is ). You can then define a context rule for this phrase, set the Windows setting to define where exactly it can be found, and then set Exclude as the Confidence change setting, to exclude this dummy API key from being flagged as sensitive data.

• Possible: If you set the Confidence Change setting to Possible, Nightfall classifies the prospective finding as an actual sensitive data and this finding is logged on the Nightfall Events page as a Possible finding (around 20-30% chance of data actually being sensitive in nature). For example, in the above scenario for the Exclude option, if you wish to get notifications even for dummy APIs, to ensure that your employees might have not shared an actual API key instead of a dummy key, you can use the Possible setting. This setting logs the finding on the Violations page as Possible finding. You can check the finding on the Violations page and if its dummy API key, and not a live API key, you can set it to Ignore the finding from the Violations page.

• Likely: If you set the Confidence Change setting to Likely, Nightfall classifies the prospective finding as an actual sensitive data and this finding is logged on the Nightfall Events page as a Likely finding (around 50-60% chance of data actually being sensitive in nature). For example, consider the scenario described above for the Exclude option. If you feel that sometimes your employees may share a dummy API key with prospective customers without using the mandatory phrase (the dummy API key is). In such cases, you they might have used some other phrases. You cannot really be sure if they transmitted dummy API keys or shared live API keys. In such cases, you can set the Confidence to be Likely (50-60% chance of data actually being sensitive). Nightfall logs it as a Finding in the Violations page with Likely confidence level. You can view the Violations page, verify this Finding which is tagged as Likely and take appropriate actions based on whether the data is actually sensitive or not.

• Very Likely: If you set the Confidence Change setting to Very Likely, Nightfall classifies the prospective finding as an actual sensitive data and this finding is logged on the Nightfall Events page as a Very Likely finding (around 80-90% chance of data actually being sensitive in nature). For example, consider the scenario described above for the Exclude option. Apart from the dummy API key there are lots of sensitive data in your organization. If an employee accidentally shares such information, they generally use phrases like our organization's API keys are, our API OAuth key is, the password is... and so on. You can create a regular expression for all such phrases, define the Window settings for them and then set the Confidence level as Very Likely. Nightfall logs such findings on the Violations page as Very Likely findings (80-90% chance of data actually being sensitive). You can navigate to the Violations page and take appropriate actions.

Once you create the regular expression for Context rule, define the Windows setting and the Confidence Change settings, you must click Save to create the context rule.

Once you save the Context rule, you can see that Nightfall creates a sentence which defines the summary of your Context rule. You can choose to edit or delete the rule, if required.

Exclusion Rules

Exclusion rules specify data that is not sensitive in nature (false positive findings). If you feel that Context rules cannot help you to stop false positive findings being logged, you can use Exclusion rules. You can also use Exclusion rules and also Context rules together to eliminate the possibility of getting false positive findings.

Nightfall provides you four methods by which you can define Exclusion rules.

Regex

You can use Regex method and define a regular expression. If a prospective sensitive data matches the defined regex, it is immediately excluded from being considered as a Finding. Nightfall also allows you to match either the whole regular expression or partial regular expression. You can use this link to generate regular expressions.

For example, consider that one of your prospective customer is testing your API and you need to share an API key for this testing. You create a series of dummy API keys (say ABCD1234, ABCD2345, ABCD3124, ABCD5412, and so on). You can create a regular expression to match each of these dummy APIs and select the Full match option. Nightfall does not treat any of these as sensitive data and excludes them.

You can observe that all the API keys start with the term ABCD. In such cases, you can use the Partial match option and just define regular expression for any one of the API keys. This is helpful if you have hundreds of dummy APIs, which have some common data between them. You need not create a regular expression for each of them.

Since all the dummy APIs start with ABCD, if Nightfall detects any of these API keys, it straight away excludes them. In this case, you must ensure that none of the live APIs have the term ABCD in them.

Dictionary

If you are facing a hard time creating regular expressions for content that needs to be excluded, you can use the dictionary option. This option allows you to define phrases that must not be considered as sensitive data. You can use this option to directly define commonly used passwords, dummy API keys, and so on.

The Dictionary exclusion rule has three options.

• Manual Entry: In this option you can directly enter the password or API key to be excluded and press enter. You can add multiple items (need not have any delimiter). You can choose to match the string partially or fully.

• Existing Dictionary: In this option, you can choose an existing Dictionary detector as the exclusion token. When you select the Existing Dictionary option, a new drop-down menu is displayed. You can select any of the previously created dictionary detector.

• Upload Dictionary: With this option, you can upload a new dictionary. The process is same as in case of dictionary detector. You can refer to the Creating Dictionary Detector document for details.

File Type

The File Type exclusion rule allows you to exclude a specific file type. If Nightfall finds that the prospective sensitive data is part of one of the file types defined in this section, it excludes it.

File type provides you two options.

• New File Type: This option allows you to define new file type(s) that must be excluded.

• Existing Custom Detector: This option allows you to use a File Type Detector created previously as the exclusion token. When you select this option, a new drop-down menu is displayed which consists of the list of previously created File type detectors. You can select the required detector.

Known Files

This option allows you to upload a file with all the data that you feel is not sensitive. Nightfall checks if any of the prospective findings match the data in this file. If a match is found, the data is not considered to be sensitive and excluded from Findings. You can also upload an existing File Fingerprint detector as the input token.

There are two options in this rule.

• Upload New File: This option allows you to upload a new file.

• Existing Custom Detector: This option allows you to use a previously created custom File Fingerprint detector.

Create Regex Detector

You can execute the following steps to create a Regex detector.

1. Navigate to the Detectors section from the left pane.

2. Click + Custom Detector and select Regular Expression.

3. Enter a name for your custom Detector in the Name field.

4. (Optional) Enter a description for the Detector in the Description field.

5. Enter the Regular expression pattern in RE2 Syntax, in the Regex field.

6. (Optional) Select the Case sensitive check box if you wish to match case sensitive pattern.

7. (Optional) Click + Rule under the CONTEXT RULES section and define regular expression pattern for Context rule.

8. (Optional) Click + Rule under the EXCLUSION RULES section and define regular expression pattern for Exclusion rule.

9. Click ADD.

This stage allows you to select detection rules to be included in the policy. The policy uses the detectors present in the detection rules to scan your data and check if there is any data leakage. You must include all the required detection rules in this section. You can select a maximum of 15 detectors in a single policy. You can use the search bar to search for a detection policy.

You must first select the required detection rule(s). Once you select the rule(s), you can set one of the following options.

• All Detection Rules: This option displays all the detection rules in the policy, irrespective of the rules selected by you.

• Selected Detection Rules: This option displays only those detection rules in the policy, that you have selected. If you move to the next stage, only the selected detection rules are included in the policy.

• Unselected Detection Rules: This option displays only those detection rules that you have not selected. You can still select any of the detection rules and even those will be excluded from the policy.

In the following image, you can view the search bar in Detection rules. You can see that:

• When you select the All Detection Rules option, all the rules are displayed.

• When you select the Selected Detection Rules option, only those detection rules are displayed that you have selected and only these rules will be used in the policy.

• When you select the Unselected Detection Rules option, only those detection rules are displayed that you have not selected. You can still select any of the detection rules and even those will be excluded from the policy.

Nightfall provides detectors for the most common data protection use cases. For unique situations, you can build custom detectors using regular expressions.

Secrets Detection

Please double-check our Nightfall Detector Glossary before creating your own, including the API and cryptographic key detectors listed below, as regex detectors can introduce noise.

Nightfall's API key Detector

Nightfall's API key supports specific detection and validation of API keys for the top 50 vendors and use cases, as shown below.

Nightfall's Cryptographic Key Detector

Nightfall's identifies popular keys for locking or unlocking cryptographic functions, including authentication, authorization, and encryption.

You can send us a request for new ML detectors directly in Nightfall.

REGEX Library

Here is a list of regex detectors used by other Nightfall customers.

If you need help with regexes or have regexes you'd like to share, please reach out to support@nightfall.ai.

In this stage, you must provide a name to your policy. Additionally, you must also select the Risk scoring method for the policy.

Naming the Policy

You must provide a name and optional description to the policy. Policy names must be unique across a single Nightfall tenant.

Risk Scoring

To help SecOps teams prioritise their efforts, we have introduced a risk-scoring system. The risk scoring system helps teams quickly identify and address the most critical issues.

This system scores violations based on various factors, such as the type and quantity of sensitive entities in a resource and the number of people with access to it.

Risk Scoring

Nightfall assigns a risk score to each violation, reflecting its potential organisational damage. The score is determined based on several factors:

• Info Type: The sensitivity level of the specific information found affects the risk score.

• Subtype: The sensitivity level of the specific subtype of information, such as vendor-specific API keys and their status, adjusts the score. For example, an active cloud infrastructure API key has a higher risk than an active mail service API key, which in turn is riskier than an inactive API key.

• Confidence: The confidence level of each finding influences the risk score.

• Count: The number of instances of each info type found in a violation impacts the score.

• Combination: The combination of findings in a violation can increase risk. For instance, a combination of patient name, date of birth, and diagnosis constitutes a HIPAA violation.

• Scope: The number of people with access to the sensitive data affects the risk score. Data shared privately, internally, or publicly will have different risk levels.

• Source: The location of the violation, such as Slack messages, GitHub repositories, Google Drive documents, or ChatGPT prompts, also adjusts the risk score.

• Document Type (Future): In the future, document types will be considered in the risk score. Leaking sensitive documents like M&A plans, executive compensation, product designs, formulas, and R&D plans can pose an existential threat to the business.

The risk score can be one of the following.

• Critical: Violations with Critical risk score pose an existential threat to the business. For example, an active AWS key can give a bad actor immediate control of the company's infrastructure. A second example would be a resource containing ≥ 100 credit card numbers or PHI records.

• High: Violations with High risk score present a significant immediate legal and financial risk to the business or a strong indicator of future critical risks. For example, if a malicious actor gains access to your customer data through a database connection string, it can result in customer churn, lawsuits, and large governmental fines. The discovery of a credential demonstrates a standard of practice that will likely lead to active and critical risk. A second example would be a resource containing 10 to 100 credit card numbers and PHI records.

• Medium: Violations with medium risk score indicate poor data hygiene, which threatens the business when discovered in numbers or combination with other entities.

• Low: Violations with low risk score indicate poor data hygiene. It may become a medium or high-risk issue when discovered with other entities. For instance, a VIN can uniquely identify a person (PII) and contribute to a protected healthcare information (PHI) finding.

Risk Scoring for Custom Detectors

To accommodate users' custom detection needs, our risk scoring system for custom regex detectors assigns scores based on the number of findings detected within documents. The predefined risk score thresholds are:

• low-risk for fewer than 10 findings,

• medium-risk for 10 to 99 findings,

• high-risk for 100 to 499 findings, and

• critical-risk for over 500 findings.

Users can override the automated score with a single, user-defined overall score, allowing for greater flexibility and control over the risk assessment process.

Custom Risk Scoring

While creating a policy, you will notice that Nightfall's advanced scoring algorithm (known as Nightfall Risk Score) is selected by default. However, you can override this with a customized score tailored to your organization's needs (known as Custom Risk Score).

Risk Scores for Historic Policies

The Risk Scoring feature is introduced in July/August 2024. All the policies created prior to this period, are automatically assigned the Nightfall Risk Score since it is the default option.

Documents with Multiple Findings of Varying Risk Scores

If a single document violates two different policies, each of which have a different level of risk score, the policy whose risk score has a higher level of precedence is assigned to the violation. For instance, consider that there are two policies called Policy A and Policy B. Policy A has a risk score High and Policy B has a risk score of Medium. If a single document violates both A and B policies, the risk score assigned to this violation is High since this score has the higher priority.

Viewing Risk Scores on Violations Page

The Violations page contains a column called Risk. This column displays the risk level for each violation.

When you hover on a risk score, you can view if the risk score is calculated manually or assigned automatically by Nightfall. If a risk score is automatically assigned by Nightfall, the label shows Nightfall Score.

If a risk score is automatically assigned manually, the label shows Custom Score.

You can also click on the label from the content preview pane of a violation to automatically search for all violations matching that label and source.

You can sort the Violations based on this column. Sorting ensures that violations are sorted in either ascending or descending order of risk levels. Security teams can sort the violations in decreasing level of severity and start remediating them.

Security teams can also search for risks with specific label (score) or source (custom or Nightfall). The risk_label operator allows you to search risks based on risk scores.

The risk_source operator allows you to filter Violations based on the source of the risk score.

Please find links to our Detection Engine FAQs below:

Detection results are comprised of two parts:

1. A data type detector that was scanned for (either a Nightfall detector or a custom regex detector). For example, a Credit Card Number.

2. A confidence that the scanned data matches the specific info type. For example, a “Very Likely” match.

Confidence may be any of the following values corresponding to the given intervals:

Each info type is unique, so it is not possible to dictate that what yields a “Very Likely” match for one detector, will yield a “Very Likely” result for another. Nonetheless, we can provide some high-level guidance. Generally speaking, higher confidence detections may have certain features that increase confidence in the detection being accurate - some examples of the features include:

• Formatting of token

• Passes validation functions

• Passes substring checks

• Context clues that are indicative of that info type

You can learn more about these specific features and how detection works in How does detection work?

Minimum confidence may be specified as part of any condition in the Nightfall Detection Engine. In the Nightfall scan API and Atlassian integrations, the confidence is returned to the end-user as part of the results. In Nightfall’s Slack and GitHub integrations, the confidence is not returned with the results at this time. You can tune the accuracy of the results by adjusting the confidence thresholds in the Nightfall dashboard.

What should I use for the Minimum Confidence setting in my Detection Rules?

For pre-built detectors, a “Possible” confidence level is triggered by the appearance of the token, without considering context, whereas “Likely” and “Very Likely” take context into account. When a custom regex is detected, its Confidence Level is assessed as “Likely” - you may determine how the Confidence Level adjusts from there based on context.

Of course, there is a tradeoff - a lower Confidence Level may result in more noise. We highly recommend setting the Minimum Confidence of every detector to Likely or Very Likely in order to reduce noise and focus your DLP efforts on priority violations. Setting your detectors to Possible or below will lead to many more findings and is best suited for scenarios in which risk tolerance is very low, or for special / advanced use cases that involve optimizing for reducing false negatives.

When setting confidence thresholds, also consider how structured the data tends to be. For example, a Social Security Number or Credit Card Number has a very typical structure and false positives may be less likely - so you could decrease the confidence threshold in order to implement a very conservative policy. On the other hand, less structured data such as Names could result in more false positives, and thus you may want to increase the confidence threshold.

Exclusion Rules, also known as an allowlist, help you reduce false positives in your sensitive findings by ignoring content from detection, or “allowing” it to pass through without being flagged.

For example, let’s say you are using the Email Address detector and you don’t want any corporate domains of “@example.com” to be detected by Nightfall. You can add “*@example.com” as a regex Exclusion Rule.

You can add in a list of known safe tokens as a dictionary or craft a regular expression. A dictionary is a list of literal values, for example, a list of dummy credit card numbers or API keys. Regular expressions are known patterns to match against, for example if you want to allow all emails of a given domain as in the example above. Regular expressions follow RE2 syntax listed . You can test your regular expression .

Exclusion Rules can be added on to any detector (either pre-made by Nightfall or off your own custom regular expression or “regex”) to omit certain “tokens” or items from resulting in detection events.

Here’s how you can build your own Exclusion Rule and attach it to a detector:

1. Select the type of Exclusion Rule you would like to use. It can be either a dictionary of words (which may be entered manually or uploaded from a list) or a regex pattern.

2. Select the match type from the dropdown menu on the right, either Partial or Full match.

3. Append your Exclusion Rule to your custom detector by clicking the "Save" button in the lower right.

Context Rules are a way to tune detectors and increase their sensitivity by up-weighting or down-weighting detection of a sensitive finding based on its surrounding context. Contrary to Exclusion Rules, which will disqualify “tokens” or items from detection, Context Rules are a way to increase or decrease the Confidence Level in your detections if there are certain types of tokens, also known as “hot words”, that surround the sensitive token.

For example, let’s say you are detecting the Social Security Number “489-36-8350”. If you see “MyCo Test SSN is 489-36-8350” you may want to lower the Confidence Level of this alert because you know this is a test or invalid SSN. Alternatively if you see “MyCo Customer SSN: 489-36-8350” you may want to upweight this confidence level because you know that at your company SSNs are real when they are formatted in this way. In this way, Context Rules help adapt detectors to specific business context relevant to you.

Context Rules are based on a regular expression (a known pattern to match against, also known as a “regex”) that you can create. They follow RE2 syntax listed . You can test your regular expression .

Context Rules can be added to any detector (either pre-made by Nightfall or to your own custom regex). You can control how many characters surrounding a finding define its context, and adjust how the original confidence level is affected by this context.

Here’s how you can build your own Context Rule and attach it to a detector:

1. Enter a regex pattern for your context rule. If you wish the pattern to be case sensitive, check the box to the right of the input window.

2. Define the window to be considered as context by specifying the number of characters away to look for your regex pattern and whether to consider context before the discovered token, after it, or both.

3. Define the confidence adjustment for your context rule from the dropdown menu. If your context rule is triggered, the finding will automatically be adjusted to your selected confidence level.

4. Add your Context Rule to the detector by clicking the "Save" button in the lower right.

Our models learn from diverse data sets, aim for high accuracy, and often perform well. However, customer data sometimes contains new patterns. Think of our models as capable students who excel in some subjects but still have unavoidable gaps in their knowledge. New data may expose gaps.

By reporting these misses, you directly help close knowledge gaps and improve alerting for your company and others.

Reporting misidentified findings

If your company is a member of our ML training program, . Your annotated sample will be added to our ML data set and used in upcoming model re-training.

If not an ML training program member, please send us anonymized samples without changing the key pattern.

Reach out to for more information about our ML training program.

If the customer needs a fix urgently, many cases can be addressed by . if you need help, please don’t hesitate to ask for help.

Reporting missed sensitive data.

Please provide us with anonymized samples without changing the key pattern.

If anonymization is impossible, coordinate with us to send the data using a secure tool like .

With the Nightfall Detection Engine, there are various ways you can improve the accuracy of detection:

1. Share your false positives with Nightfall We'll identify the issue and retrain our models. If your company is a member of our ML training program, . Your annotated sample will be added to our ML data set and used in upcoming model re-training. If not an ML training program member, please send anonymized samples without changing the key pattern. Reach out to for more information about our ML training program.

2. Modify the detector using an exclusion rule recognizing known test or mock values. Read more about exclusion rules in

3. Modify the detector using a context rule recognizing common patterns in the text surrounding the sensitive token. Read more about context rules in

4. Increase the detector's minimum confidence to “Likely” or “Very Likely”. Read more about detector confidence levels in

5. Set up the Detection Rule’s to trigger when each entity type is found. Common Entities such as names, addresses, phone numbers, and emails are found everywhere and are generally not sensitive unless found in combination.

6. Set up policies appropriate to the scanned data source - i.e. integration, channel, folder, instance, etc. For example with Slack Enterprise, you can enable or disable detection in private channels, direct messages by channel ID, workspace ID, or in all private channels & DMs. Reach out to and we will help.

7. Increase the Detection Rule’s minimum number of findings threshold. For example, when trying to detect phone list sharing, set you detection rule to ony trigger alerts when seeing 10 or more person names and phone numbers.

Previously, Nightfall Password detector used to report Zoom passwords as sensitive data findings. However, the Nightfall Password detector now recognizes passwords found in personal Zoom links. Customers sharing these links in docs and messages will no longer be alerted for these non-sensitive findings.

The detector can now identify patterns such as:

• Zoom meeting invites containing embedded PWD parameters and separate alpha-numeric passcodes.

• Zoom recording URLs with associated passcodes.

Examples:

1. Multi-line meeting invite with embedded Zoom password and passcode

   ./j?pwd=VBhZM3h5SEhQTXEzQzEyYUREWkJHQT09

   Meeting ID: 991 1959 8432

   Passcode: 475749

2. Recording URL with passcode

   ./share/zuBQvxkbLce3PRl6eYSBXN0Uj47mE9grL_6G4J1qeFcaWCqB94hdgC5hb4B3R3SF.VxapNL9oXfGuiokO

   Passcode: BH%$!.06

An Event is an incident that is triggered when a data leak is detected by Nightfall. The Data Sensitive Events page displays all the details about the data sensitive events recorded in Nightfall from all the integrations.

Before we proceed further, let's understand what exactly is a Finding and a Event.

Finding: A Finding refers to a single instance of a data leak incident detected during scanning.

Event: An Event is an incident of data leaks, and a collection of findings that were detected while scanning an entire document or data in your database.

Events vs Finding

When Nightfall scans data, a single Event can have multiple findings. For instance, consider that you upload a document to Google Drive. This document has 10 instances of secret information (10 API keys or 10 credit card numbers). These 10 instances are known as Findings. Since Nightfall scans the entire Google document at once, these 10 Findings are recorded as a single Event.

Event List View

The sensitive data Event list view page consists of a table that displays the details of each Event. The contents consist of columns which are described as follows.

Event Likelihood

The Finding column displays the name of the detector whose condition was violated thus resulting in the generation of the Event. The Finding column displays a list of Events that originated from scanning a single file, a web page, or any other publicly available entity.

Each Event's name is the same as the Detector's name whose condition was violated. You can also view the Likelihood or confidence level (Possible, Likely, Very Likely) of an Event being an actual case of data leak and also the number of instances for each Likelihood. The Possible Likelihood Findings are displayed in blue color with a fully dotted blue circle. The Likely Finding is displayed in yellow color with a half-dotted and half-shaded yellow circle. The Very Likely Finding is displayed in red color with a red circle.

For instance, in the following image, a detector called Person name violated 28 times. Of these 28 instances of violations, 1 is Possible Event (in blue), 5 are Likely Events (in yellow), and 22 are Very Likely Events (in red).

Event Detail View

The Event detail view consists of various sections. These sections are as follows.

Event Detail View First Section

Each field is highlighted and a number is associated with the highlighted field. Using the number, you can refer to the section that follows the image to learn more about the field associated with that number.

1 - Name of the Event

3 - The nature (Credit card in the above image) and number of Findings (1 in the above image) in the Event.

4 - The name of the document in which the sensitive data was found (TEst Automation Action.Docx in the above image). If there are multiple documents that contain sensitive data, each document is listed here. However, in this case only one document contained sensitive data.

6 - Number of Findings in the selected document. In the above case there is only one finding and it has a Likelihood is Possible.

Event Detail View Second Section

The contents of this section vary for each integration. You can view the complete list of fields available in the second section after you click the Expand Details button.

The following section contains tabs for each integration. Each tab represents an integration and contains an image of the second section followed by the description of each fields.