Finding patience in the two-week data import

I've been working on a new feature for Repeat Customer Insights over the past couple of months.

I'm not ready to announce it yet, I hate when companies talk about things that never get released, but the process has illustrated a few things.

The first of which is the importance of patience.

The feature needs new data from Shopify which means having to import something like 80 million records of data from them. But due to how you get data from Shopify, it's S L O W.

Even with everything in the app scaled up to Black Friday levels of performance, it's going to take about two weeks to get all of the data loaded.

I could rush it, as I have done in the past, but that risks errors and means late nights baby-sitting the data.

Some things just can't be rushed and just take time.

I've seen this in code that works with large sets of data.

I've seen it with SEO.

I've seen it with content marketing.

After you've done everything you should have reasonably done, sometimes the only thing left is patience. Give things time to finish. Find something else to work on meanwhile.

That still won't stop me from refreshing my import progress every couple of hours though. Hey, I never said I'm perfect.

Eric Davis