Bounded serverless crawling
SQS buffers audit work and Lambda concurrency limits control crawler capacity. Each job enforces page, byte, redirect and duration limits, while callback credentials allow only the expected audit result to be returned.
Architecture and workflow
Your website is scanned away from the customer-facing API.
A submitted public URL is validated and queued before an isolated AWS Lambda worker discovers eligible pages, respects robots.txt, evaluates the site and returns a signed result to the application API.
Safe public-site analysis
Private and cloud-metadata addresses are rejected, redirect targets are revalidated and the crawler never authenticates, submits forms, exploits vulnerabilities or performs penetration testing.