Senior Data Infrastructure Engineer


About the Team We’re live and breath big data. On a daily basis, we ingest and extract useful information from hundreds of live TV channel as well as collect, analyze and report on information from millions of TVs. We’re all in with AWS and make extensive use of many of their services. On any given day we are running between 2000-2500 instances with a stack and application set that includes Ubuntu, C, Python, Go, Kinesis, RDS, Redshift and Stackdriver. We are currently using data structures, processing and reporting approaches that got us this far but we’re almost positive are in need of a rethink and redesign. Our current approach involves large AWS Redshift clusters, numerous ETL processes with business logic in those ETLs as well as very complicated and lengthy SQL queries to produce reports that are run hourly and which often contain largely redundant data. . What You Will Do Initially you would assume operational ownership of our current databases and clusters (we have a long-standing relationship with an organization that provides us with support.) As part of this you will be doing a deep dive into all things data-related including but not limited to: how data is structured, how if flows through the system, how it is accessed, how we currently manage our databases and data stores as well as their overall status and health. You would support the architecture, creation and operation of the next generation data infrastructure. About You You have significant experience working in 24x7 customer-facing production environments and are willing to work on rotation. You have a deep background with monitoring/alerting, configuration management, automation as well as integration and deployment pipelines. You are proficient in at least one programming language (Python preferred but not required). Ideally, you have good AWS experience with at least some of their core infrastructure-related tools and services such as IAM, S3, SQS, Cloudwatch, and Cloudtrail but we are flexible on this if you are new to AWS.You are someone that people come to when they need help because they know you want to. You strive to understand the problem you are trying to solve before deciding on the solution, and you are thoughtful and methodical in its implementation vs. using shiny new untested tech