In his article, David Coker discusses the lessons learned when the School of Finance and Accounting built a tick-by-tick database to capture US equity prices to help improve subsequent phases of database expansion and development.
He wrote: “The first problem was the difficulty of rapidly expanding databases stored on traditional platforms. Our solution was to migrate to the cloud, where nimble providers allowed us to rapidly expand our resource footprint on demand.”
He also said they reused old code to save time, and added: “Since we had aggressive development deadlines we reused existing code as much as possible.
“Rather than creating new Python programs to deliver ETL, we repurposed existing Perl programs with slight modifications rather than develop from scratch.”
He also spoke about the win-win scenario for students during this experience, as class lessons were augmented with project work, so students help populate the SFA Fintech Research Database.