This repository was archived by the owner on May 28, 2024. It is now read-only.
Filter out provisional data #206
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR omits provisional data from the DO data that is used for model training. Two new targets are added to subset the data downloaded from NWIS (
p2_daily_data_filteredandp2_inst_data_filtered). I also updated downstream viz targets to use the new filtered data targets.@jsadler2 We can either merge this or not. In practice, by updating⚠️ All targets downstream of
latest_dateto2021-10-01and with our current group of sites, the subset data are not different from the downloaded data because there are no provisional records. So these changes really just serve to codify our decision to not use provisional records in the code base.p2_daily_combinedwould rebuild, which includes most if not all data prep targets.