Uploading and Scanning API Calls
Last updated
Was this helpful?
Last updated
Was this helpful?
Nightfall's upload process is built to accommodate files of any size. Once files are uploaded, they may be scanned with and to detect potential violations.
Many users will find it more convenient to use our our to complete the upload process.
Uploading files using Client SDK libraries requires fewer steps as all the required API operations are wrapped in a single function call. Furthermore these SDKs handle all the programmatic logic necessary to send files in smaller chunks to Nightfall.
For users that are looking to understand the entire end-to-end, that is also outlined in this document. We will walk you through the order of operations necessary to upload the file.
Rather than implementing the for the upload functionality yourself, the Nightfall’s provide a single method that wraps the steps required to upload your file.
Below is an example of uploading a file from our and our .
To run the node sample script you must compile it as TypesScript. Save it as a .ts file and run
tsc <yourfilename>.ts -lib ES2015,DOM
You can then run the resulting JavaScript file:
NIGHTFALL_API_KEY=<YourApiKey> node yourscriptname.js
Note that these examples use an email address to receive the results for simplicity.
The upload process consists of 3 stages:
POST /v3/upload
As part of the initialization you must provide the total byte size of the file being uploaded.
You may also provide the mime-type, otherwise the system will attempt to determine it once the upload is complete.
The id
of the returned JSON object will be used as the fileId
in subsequent requests.
The chunkSize
is the maximum number of bytes to upload during the uploading phase.
The size of these chunks are determined by the chunkSize
value returned by POST /upload
endpoint used in the previous step.
Below is a simple example where the file is less than the chunkSize
so may safely be uploaded with one call to the upload endpoint.
If your file's size exceeds the chunkSize
, to upload the complete file you will need to send iterative requests as you read portions of the file's contents. This means you will send multiple requests to the upload
endpoint as shown above. As you do so, you will be updating the value of the X-Upload-Offset
header based on the portion of the file being sent.
Each request should send a chunk of the file exactly chunkSize
bytes long except for the final uploaded chunk. The final uploaded chunk is allowed to contain fewer bytes as the remainder of the file may be less than the chunkSize
returned by the initialization step.
The request body should be the contents of the chunk being uploaded.
The value of the X-UPLOAD-OFFSET
header should be the byte offset specifying where to insert the data into the file as an integer. This byte offset is zero-indexed.
Successful calls to this endpoint return an empty response with an HTTP status code of 204
POST /v3/upload/<uploadUUID>/finish
When an upload completes successfully, the returned payload will indicate the mimeType the system determined to file to be if it was not provided during upload initialization.
Once a file has been marked as completed, you may initiate a scan of the uploaded file.
After an upload is finalized, it can be scanned against a Detection Policy. A Detection Policy represents a pairing of:
a webhook URL
a set of detection rules to scan data against
You may also supply a value to the requestMetadata
field to help identify the input file upon receiving a response to your webhook. This field has a maximum length 10 KB.
Nightfall will verify that the webhook URL is valid before launching its asynchronous scan by issuing a challenge.
Below is a sample Python script that handles the complete sequence of API calls to upload a file using a path specified as an argument.
You may also want to use a webhook. See for additional information on how to set up Webhook server to receive these results.
Once the upload is complete, you may initiate the
After we discuss each API call in the sequence, you will find a script that walks through the at the end of this guide.
The first step in the process of scanning a binary file is to initiate an upload in order to get a fileId
through the Initiate a .
Use the endpoint to upload the file contents in chunks.
See the below for an illustration as to how this upload process can be done programmatically.
Once all chunks are uploaded, mark the upload as completed using the .
The scanning process is asynchronous, with results being delivered to the webhook URL configured on the detection policy. See for more information about creating a Webhook server.
Exactly one policy
should be provided in the request body, which includes a webhookURL
to which the callback will be made once the file scan has been completed (this must be an HTTPS URL) as well as a Detection Rule as either an a or as a rule that has been .