Initiate Crawl API

The Crawling Initiating API lets users submit URL with customizable parameters to start a web crawler. This crawler navigates the URL, parses the content, and extracts data based on specific rules

Initiate crawl

POST https://secctrl.tutelar.io/api/v1/web-crawling/website-crawl

401 : Unauthorized

{ "message": "Invalid authentication credentials" }

Post parameters description is given below,

Post Parameter	Type	Is Mandotory	Description
website	String	Yes	For Crawling The Website, Need To Give Absolute URL of The Website
level	String	Yes	Mention The Level Of Crawling Like Basic/Advance
merchant*	Object	Yes (trade_name only Mandatory)	For Identify The Merchant Details & Merchant Business Details
address	String	No	Input value of crawling website address
ucic	String	No	Unique customer identification code used to track and identify crawled websites.

Sample post parameters

{
    "website":"https://www.tutelar.io",
    "level" : "basic",
    "merchant" : {
        "id" : "TUTMERCH001",
        "trade_name": "tutelar pvt ltd"
    },
    "address":"chennai",
    "ucic": "WEB123" 
}

Success Response

{
    "success": true,
    "code": 200,
    "message": "Queue added",
    "data": {
        "status": "in_progress",
        "caseId": "gKXvLK9Y7gPAmyYCxGDn3p",
        "websiteName": "tutelar.io",
        "merchant": {
            "id": "TUTMERCH001",
            "trade_name": "tutelar pvt ltd"
        },
        "createdAt": "2024-01-05T09:40:58.157Z"
    }
}

Failure Response

{
    "success": false,
    "code": 300,
    "message": "Please provide the exact URL of the Website",
    "data": {}
}