Initiate Crawl API

The Crawling Initiating API lets users submit URL with customizable parameters to start a web crawler. This crawler navigates the URL, parses the content, and extracts data based on specific rules

Initiate crawl

POST https://secctrl.tutelar.io/api/v1/web-crawling/website-crawl

401 : Unauthorized

{ "message": "Invalid authentication credentials" }

Post parameters description is given below,

Post Parameter	Type	Is Mandotory	Description
website	String	Yes	For Crawling The Website, Need To Give Absolute URL of The Website
level	String	Yes	Mention The Level Of Crawling Like Basic/Advance
merchant*	Object	Yes (trade_name only Mandatory)	For Idintify The Merchant Details & Merchant Business Details

Sample post parameters

{
    "website":"https://www.tutelar.io",
    "level" : "basic",
    "merchant" : {
        "id" : "TUTMERCH001",
        "trade_name": "tutelar pvt ltd"
    }
}

Success Response

{
    "success": true,
    "code": 200,
    "message": "Queue added",
    "data": {
        "status": "in_progress",
        "caseId": "gKXvLK9Y7gPAmyYCxGDn3p",
        "websiteName": "tutelar.io",
        "merchant": {
            "id": "TUTMERCH001",
            "trade_name": "tutelar pvt ltd"
        },
        "createdAt": "2024-01-05T09:40:58.157Z"
    }
}

Failure Response

{
    "success": false,
    "code": 300,
    "message": "Please provide the exact URL of the Website",
    "data": {}
}