Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DEV-11036: Initial Run - No Initial Copy Bug #4245

Open
wants to merge 4 commits into
base: qat
Choose a base branch
from

Conversation

dpb-bah
Copy link
Contributor

@dpb-bah dpb-bah commented Dec 20, 2024

Description:
The Initial Run w/ "No Initial Copy" still does some initial copy with raw.transaction_normalized which includes questionable data Pre Phase 3. This simply says to act as if raw.transaction_normalized doesn't exist if there's no initial copy.

Technical details:
N/A

Requirements for PR merge:

  1. Unit & integration tests updated
  2. API documentation updated
  3. Necessary PR reviewers:
    • Backend
    • Domain Expert
  4. Matview impact assessment completed
  5. Frontend impact assessment completed
  6. Data validation completed
  7. Confirmed the missing data appears after a full reload with no initial copy w/ this code.
  8. Appropriate Operations ticket(s) created - https://federal-spending-transparency.atlassian.net/browse/DEV-11858
  9. Jira Ticket DEV-11036:
    • Link to this Pull-Request
    • Performance evaluation of affected (API | Script | Download)
    • Before / After data comparison

Area for explaining above N/A when needed:

6. Initial data validation was successful, not a full regression check of the data though.
9. Regarding performance, the initial run with no initial copy took minutes and the initial pipeline run took ~4hrs loading in all the data from scratch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants