Senior Data Engineer (Customer Data Products) - National Board of Medical Examiners

Overview

Any additional information you require for this job can be found in the below text Make sure to read thoroughly, then apply.

In this Data Engineer role, you’ll help modernize NBME’s data platform by building scalable data pipelines, lakes, and feature stores on AWS to support analytics, AI/ML, and decision-making. You’ll collaborate with data scientists, architects, and stakeholders to prepare data for AI applications and lead initiatives introducing modern data techniques. The position blends hands-on engineering with leadership in data engineering practices within a mission-driven healthcare education context.

Compensation / Benefits
• Healthcare, Dental, Vision plans
• 401(k) with match
• Tuition Reimbursement Plan
• Commuter Benefit for transit/parking
• Remote Friendly Workplace

Responsibilities
• Develop and manage cloud-based data pipelines, feature stores, and vector databases aligned with security and lifecycle best practices
• Collaborate with data scientists, AI researchers, data/enterprise architects, and stakeholders to prepare data for AI/ML model training and feature extraction
• Lead evaluation and experiments with batch/streaming data, LLM prep frameworks, and MLOps tools
• Act as technical lead to promote AI/data technologies across the organization
• Contribute to defining and refining data engineering processes and procedures
• Educate ETL developers on cloud-based data initiatives and practice
• Ensure data integrity, security, and high data quality for Data Services, Analytics, and MDM
• Coordinate solution designs, development, testing, and delivery of data solutions
• Build automated, scalable, test-driven data pipelines
• Utilize Git, CI/CD, and release management to enhance AWS pipelines
• Collaborate with Data Engineers, DevOps, and architects on DataOps improvement opportunities

Key requirements
• Bachelor's degree
• 7+ years of application development exp
• 4+ years in big data technologies xklufbp
• 4+ years of AWS cloud experience
• 4+ years in Python, SQL, Scala, or Java
• 4+ years distributed data tools (MapReduce, Hadoop, Hive, EMR, Kafka, Spark, MySQL)
• 4+ years real-time data/streaming applications
• 4+ years NoSQL (Mongo, Cassandra)
• 4+ years data warehousing (Redshift)
• 6+ years UNIX/Linux, shell scripting
• 7+ years Agile practices
• 7+ years SQL optimization
• 4+ years PySpark
• 3+ years orchestration (AirFlow, Kubeflow, AWS Step Functions, Luigi)
• collaboration
• leadership
• communication
• AWS cloud
• Python
• SQL