Optimise bitcoin export so it doesn't query all partitions every day

[medvedev1088]

Right now load_dag scans data in all partitions every day. In particular enrich transactions sql https://github.com/blockchain-etl/bitcoin-etl-airflow/blob/master/dags/resources/stages/enrich/sqls/transactions.sql needs to join inputs and outputs and requires scanning all past data.

An alternative is to enrich transactions in export_dag using https://github.com/blockchain-etl/bitcoin-etl#enrich_transactions.

This will reduce the BigQuery costs significantly.

---

This might also require changing timestamp field type from int to iso 8601 in export jobs (breaking compatibility change so will bump the version to 2.*), so that raw tables can be partitioned by this field. Now the raw tables are not partitioned which makes the enrich job scan whole table.

[sfsf9797]

hi @medvedev1088, just checking if this issue is still open since it has been a while since it was created.

Anyway, I think I might be able to contribute to this by:

1. add the enrich _transactions command into the build_export_dag.py
2. remove the part of the code that does enrich in build_load_dag.py and make some other necessary changes.

finally, I would validate the results!
what do you think? please let me know if any concerns or suggestions. thanks!