Welcome to Spark

Spark is a trustless protocol for sampling retrievals from Filecoin Storage Providers.

When Filecoin Storage Providers (SPs) accept deals from the FIL+ program for large data sets (FIL+ LDN), there is an agreement to make all the data readily retrievable on the network. In addition to storing the sealed copy of the data, SPs must keep another “hot copy” to provide fast retrievals without having to unseal the sealed copy first.

However, the core Filecoin protocol does not enforce this agreement to make data readily retrievable. As a result, only a tiny fraction of data stored in LDN FIL+ deals can be publicly retrieved today.

To improve this situation, we first and foremost need more data about content retrievability. However, measuring whether Filecoin data is publicly retrievable poses several tough challenges:

• Large scale: Filecoin has over 1 million gigabytes worth of storage deals, over 47 million active deals, and over 15 million Piece CIDs. Measuring retrievals of a meaningful subset of such a large dataset requires a lot of computing power and network bandwidth.
• Trust: How can the community trust the measurements? Who operates the checker nodes, what protocol is followed by the network, and how well can the system detect fraud?
• Funding: Who is going to
• pay for running the nodes orchestrating the work,
• compensate for the resources consumed by the nodes performing retrieval checks,
• fund the teams building these tools?

Spark aims to solve these challenges, provide trusted metrics about retrievals performed from real user networks, and ultimately drive improvements in the retrievability of data stored on Filecoin.

Handle Large Scale

SPARK leans on a decentralised network permissionless checker nodes. Anybody can run a SPARK node, perform retrieval checks and earn rewards for doing so.

Running a SPARK node is super easy, thanks to Filecoin Station - a desktop app aimed at non-technical users. There’s nothing to configure, just launch the app, and you’re all set.

Instil Trust

We put many anti-fraud features into the SPARK protocol. Each one makes it more and more difficult for node operators to cheat.

• We use IPv4 addresses as a scarce resource, making Sybil attacks too expensive.
• The network performs retrieval checks with a high redundancy. This allows us to compare results submitted by different nodes and build committees large enough to give us strong confidence in an honest majority.

The orchestration layer has a decentralised design, allowing multiple interested parties to run the network components.

The SPARK protocol is highly verifiable. The data used for evaluation and fraud detection is publicly available and committed to on chain. The evaluation and fraud-detection process is deterministic - anybody can re-run the algorithm on public data to get the same results as we did.

Provide Funding

SPARK leverages Meridian to handle rewards for node operators. Meridian is a smart contract running on FEVM, which evaluates the impact of individual nodes verifiably. To fund Spark, funders can drop rewards into the Spark smart contract, governed by the Meridian Framework.

Funding is arguably the most challenging part. There have been many attempts to build a permission-less network of incentivised nodes. Most (if not all) of them got minimal traction because they could not secure sustainable funding.

The Station (Spark) Team is looking into ways to link Spark to Filecoin block rewards and QAP in order to incentivise Storage Providers to provide a good retrieval service, as well as create a long term and sustainable future for Spark funding.

There are also early discussion about a Spark token as an alternative funding strategy.