Page MenuHomePhabricator

ores_predictions_daily DAG fails to overwrite files
Closed, DeclinedPublic

Description

The fetch_articletopic_prediction_thresholds and fetch_drafttopic_prediction_thresholds initially failed on june 16, 2022 for unknown reasons, and then continued to fail because the first execution left behind an empty destination file and the system refused to overwrite it. Ideally this should auto-magically fix itself and overwrite the invalid file. We already allow parquet datasets to be overwritten, we should be able to do soemthing similar here.

Event Timeline

Gehel triaged this task as High priority.Jun 27 2022, 3:27 PM
Gehel moved this task from needs triage to ML & Data Pipeline on the Discovery-Search board.
dcausse subscribed.

fetch_articletopic_prediction_thresholds has been removed