Testing a pipeline
Join a month of New York's cab trips to the taxi zones and to Central Park's weather with Spark, and test the pipeline that does it: NOAA's daily weather file read into units, each borough's trips on every day of August, wet days against dry days of the same weekday, a review of an assistant's daily table, and tests of your own that catch the mistakes planted in the pipeline.
- You build
- weather.py, then daily.py, then rain.py, then daily_table.py, then test_pipeline.py
- Stages
- 3 Build, 1 Catch, 1 Ship
- Time
- About 3 hours
- Runs on
- Your machine
Loading the workspace…
Loading the workspace…