Overview
You will design and build data pipelines that transform raw data into actionable insights for national security clients. You’ll handle diverse sources—batch, streaming, real-time, and unstructured—and apply distributed compute to scale. You’ll work with cross-functional teams to ensure data quality, security, and reusable code, while supporting the scrum process and maintaining existing data infrastructure. This role combines hands-on engineering with data governance and collaboration to deliver impactful analytics.
Responsibilities
- Design and develop data pipelines, ingestion, and orchestration tooling
- Create database schemas and model data for consistent integration
- Integrate and enrich data from multiple sources ensuring quality
- Design and implement ETL processes, including NiFi
- Write reusable, open-by-default code and maintain existing components
- Collaborate with operations to diagnose issues with integrated data
- Translate user needs into actionable epics and stories for the scrum team
- Implement data security measures to protect sensitive information
- Monitor data systems for performance and perform necessary updates
- Adopt a test-driven approach to clean, secure coding
Key requirements
- Experience building data pipelines and data ingestion solutions
- Proficiency with ETL tooling and NiFi
- Database schema design and data modeling
- Data integration from diverse sources and data quality focus
- Knowledge of data security practices
- Ability to decompose work into epics and stories for agile teams
- Strong coding practices with emphasis on maintainability and security
- Experience monitoring and maintaining data systems
- collaboration
- communication
- problem-solving
- NiFi
- ETL design and implementation
- data pipeline development
…
