🚄Backfilling

Run your scripts for historical events, for testing or initial fetching of your data

The backfilling procedure executes your custom processor script for all events that has happened in the past. You can do this for the purpose of testing your scripts (e.g. only running against 100 blocks), or for the purpose of creating a fresh database for all your previous events.

That would be as follows:

1. You develop and deploy your custom scripts.

2. Run backfill command against certain block range (e.g. from the deployment of your contract).

3. See the logs printed by your processor.

4. Query the data via SQL.

Depending on your cluster size backfilling can handle ~100,000 blocks / second. This excludes your custom processor duration.

A real experiment for 2 million events, about 4 years worth of blocks, took about 30 minutes to finish.

If you have large backfilling needs reach out to our engineers @FlairSDK or https://discord.gg/flair.

flair backfill command

Learn about different ways of backfilling historical data using flair backfill command.

Example of backfill command, and a broken custom processor which was fixed mid-way, see "processor retries" and "recent logs".

flair logs command

You can see logs printed by your custom processor during a backfill job. You can even narrow down the logs to a specific "TransactionHash" or "BlockNumber" to debug much faster.