Duties:
Meet with external vendors to negotiate and create bespoke programmatic data feeds from them. Typically these will be JSON, XML, or SQL dumps.
Manage sftp and Amazon S3 buckets to facilitate data transfers.
Create and adjust data schemas to accommodate.
Script in python and SQL to ingest and process data on a regular basis.
Create dashboards showing status of the operation.
Requirements:
Python scripting
SQL programming
Helpful:
Experience with Hive
Familiarity with Amazon S3
