Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

[11:43:29 CST(-0600)] <TonyUnicon> as opposed to just the first

[11:44:37 CST(-0600)] <TonyUnicon> now im confused

[11:44:54 CST(-0600)] <TonyUnicon> when we spoke about in memory validation before we write

[11:45:05 CST(-0600)] <TonyUnicon> I thought you meant before we write to the staging tables

[11:45:29 CST(-0600)] <dmccallum54> raw files -> per-row type, width, nullity validations -> filtered/validated files -> batched inserts to stage tables -> upserts to live tables