Create a Prompt-based File Classifier
Define a file classifier using natural language. No training required.
Getting Started
Nightfall provides 20+ pre-built file classifiers out of the box, covering common sensitive document types such as contracts, background checks, financial records, and HR documents.
A Prompt-Based File Classifier is designed for cases where:
Your document type does not fit one of Nightfall’s existing classifiers, or
You need more specificity than a general-purpose classifier provides
This is especially useful for customer-specific document types that:
Follow a consistent structure
Represent a distinct business workflow
Are sensitive due to their intent, even if they vary slightly in format
A Prompt-Based File Classifier allows you to describe these document types using natural language, enabling Nightfall to classify entire files based on intent, structure, and meaning.
Unlike entity detectors, which identify individual data elements, file classifiers answer the question:
“What kind of document is this?”
Prompt-based file classifiers are best suited for:
Organization-specific or proprietary document types
Business-critical workflows (e.g., custom loan applications, internal risk assessments)
Insider-risk and exfiltration monitoring
Documents where sensitivity comes from context, not individual entities
File classifiers operate on files and attachments and evaluate the entire document, not individual fields or messages.
To create a prompt-based entity detector, navigate to Detection → Detectors, then click + Custom Detector in the upper-right corner and select File Classifier (Prompt-Based) from the menu.
This opens the Prompt-Based Entity Detector design tab, shown below.
How Nightfall Interprets File Classifier Prompts
Prompt-based file classifiers use an LLM to evaluate the entire document and determine whether it matches the document type you describe.
Rather than detecting specific entities, Nightfall considers:
The overall structure of the document
The types of information present
How sections relate to each other
Whether the document’s intent matches your description
Accuracy improves when you clearly describe what the document is, what it typically contains, and what should not be classified as a match.
Understanding Prompts, Keywords, and Sample Files
Prompt-based file classifiers use three primary inputs:
Prompt – describes the type of document to detect
Keywords – important terms commonly found in the document
Sample File – a sanitized, representative example of the document. A template or form is OK.
Together, these inputs help Nightfall reliably classify documents with high precision.
Prompt
The Prompt describes the document type you want to classify. It should focus on intent and structure, not individual sensitive entities.
A strong prompt typically includes:
Document type (what the file represents)
Purpose of the document
Key sections or information types
Typical use cases or workflows
What should not be considered this document type
Prompt example
Custom Mortgage Application
Detect a mortgage application form used by a lending organization. This document collects borrower personal information, employment details, income, assets and liabilities, property information, and loan terms. It is used during the mortgage origination process and should not include finalized loan agreements or closing disclosures.
Keywords
Keywords help reinforce classification by highlighting terms commonly found in the document.
Keywords should:
Be comma-separated
Reflect section headers, labels, or repeated concepts
Be descriptive but not overly specific
Example keywords
Custom Mortgage Application
Sample File
The Sample File provides Nightfall with a concrete example of the document type you want to classify.
Best practices for sample files:
Use a realistic but sanitized document
Include representative sections and structure
Avoid production secrets or live customer data
Upload a single, complete document (PDF, XLSX, etc.)
The sample file is used only to guide classification and improve precision.
Test Prompt
Once you are satisfied with the classifier definition, click Test Prompt to run a prompt hygiene check.
During this check, Nightfall evaluates the Prompt, Keywords, and Sample File together to confirm that the classifier can accurately identify the document type and meet Nightfall’s detection quality standards.
If the prompt passes the hygiene check, the file classifier can be:
Added to your Custom Detectors list
Used in a Detection Rule
Included in a Policy for monitoring, alerting, or enforcement
If the prompt does not pass, Nightfall will guide you to improve it by:
Clarifying the document intent
Adjusting keywords
Providing a more representative sample file
This ensures that file classifiers are precise, reliable, and safe to deploy in production environments.
Sample Prompt and Files
Last updated
Was this helpful?