Revamp the entire data import process

The current process of throwing all of our data imports into a single queue is ridiculous. It creates a massive bottleneck where reasonably sized files take up to 4 hours just to START being processed. It means we have to wait an unreasonable amount of time to validate a small, test file (i.e. 4 or 5 rows) so that when we do upload the full file, we know it's going to be successful.


The wait times are especially bad during the first couple of weeks of the school year. The explanation that is typically given is, "Well, it's just really busy this time of year." That is an unacceptable answer. If you KNOW it's going to be busy at certain times every year, then do something about it. Prepare for it. Don't just throw your arms in the air and say, "It is what it is." Scale up to meet the demand!

  • john simpson
  • Aug 26 2021
  • Attach files