Stanford University is seeking a Big Data Architect 1 for a 1 year fixed term (possibility of renewal) to design and develop applications, test and build automation tools and support the development of Big Data architecture and analytical solutions.
About Us:
The Department of Biomedical Data Science merges the disciplines of biomedical informatics, biostatistics, computer science and advances in AI. The intersection of these disciplines is applied to precision health, leveraging data across the entire medical spectrum, including molecular, tissue, medical imaging, EHR, biosensory and population data.
About the Position:
We are seeking an experienced ML Data Engineer to drive the programmatic curation, cleaning, and generation of healthcare data. In this role, you will focus exclusively on developing and maintaining automated, ML-accelerated pipelines that ensure high-quality data ready for machine learning applications. Your work will be pivotal in shaping the integrity of our data and supporting downstream predictive models in a complex healthcare environment.
You Will Find This Position a Good Fit If:
- You are passionate about transforming raw healthcare data into valuable insights.
- You believe in the critical role of robust data curation in advancing machine learning in healthcare.
- You thrive in environments where you can work independently on complex data challenges while collaborating with multidisciplinary teams.
- You are excited to work with patient-level data and embrace challenges related to data diversity and complexity.
Duties include:
- Design Big Data systems that are scalable, optimized and fault-tolerant.
- Work closely with scientific staff, IT professional and project managers to understand their data requirements for existing and future projects involving Big Data.
- Develop, test, implement, and maintain database management applications. Optimize and tune the system, perform software review and maintenance to ensure that data design elements are reusable, repeatable and robust.
- Contribute to the development of guidelines, standards, and processes to ensure data quality, integrity and security of systems and data appropriate to risk.
- Participate in and/or contribute to setting strategy and standards through data architecture and implementation, leveraging Big Data, analytics tools and technologies.
- Work with IT and data owners to understand the types of data collected in various databases and data warehouses.
- Research and suggest new toolsets/methods to improve data ingestion, storage, and data access.
Key Responsibilities:
- Data Pipeline Engineering:
○ Design, implement, and maintain robust pipelines for the programmatic cleaning, transformation, and curation of healthcare data.
○ Develop automated processes to curate and validate data, ensuring accuracy and compliance with healthcare standards (e.g. OMOP CDM, FHIR).
○ Leverage core machine learning techniques to generate datasets, clean existing health records, join heterogeneous data sources, and enhance data quality for model training.
○ Implement innovative solutions to detect and correct data inconsistencies and anomalies in large-scale healthcare datasets.
- Healthcare Data Expertise:
○ Work extensively with patient-level health data, ensuring that data handling practices adhere to industry regulations and ethical standards.
○ Utilize the OMOP Common Data Model (OMOP CDM) to standardize and harmonize disparate healthcare data sources, enhancing interoperability and scalability.
- Collaboration & Continuous Improvement:
○ Collaborate closely with data scientists, clinical informaticians, and engineers to align data engineering practices with analytical and clinical requirements.
○ Continuously monitor, troubleshoot, and optimize data workflows to support dynamic research and operational needs.
The expected pay range for this position is $157,945 to $177,385 per annum.
Stanford University provides pay ranges representing its good faith estimate of what the university reasonably expects to pay for a position. The pay offered to a selected candidate will be determined based on factors such as (but not limited to) the scope and responsibilities of the position, the qualifications of the selected candidate, departmental budget availability, internal equity, geographic location and external market pay for comparable jobs.
At Stanford University, base pay represents only one aspect of the comprehensive rewards package. The Cardinal at Work website (https://cardinalatwork.stanford.edu/benefits-rewards) provides detailed information on Stanford’s extensive range of benefits and rewards offered to employees. Specifics about the rewards package for this position may be discussed during the hiring process.
Consistent with its obligations under the law, the University will provide reasonable accommodations to applicants and employees with disabilities. Applicants requiring a reasonable accommodation for any part of the application or hiring process should contact Stanford University Human Resources at stanfordelr@stanford.edu. For all other inquiries, please submit a contact form.
Stanford is an equal employment opportunity and affirmative action employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, protected veteran status, or any other characteristic protected by law. Stanford welcomes applications from all who would bring additional dimensions to the University's research mission.
DESIRED QUALIFICATIONS:
- 3+ years of experience in software development and data engineering with a strong focus on data cleaning, transformation, and creation.
- Proficiency in Python and experience with data processing libraries (e.g., Pandas, Polars, NumPy).
- Hands-on experience in building and maintaining automated data pipelines for large-scale data processing.
- Familiarity with machine learning frameworks (e.g., PyTorch, JAX, scikit-learn) as applied to data quality and augmentation tasks.
- Expertise in working with healthcare data, including familiarity with the OMOP Common Data Model (OMOP CDM).
- Strong experience in a Linux environment and comfort with UNIX command-line tools.
- Proven ability to work collaboratively in multidisciplinary teams and communicate technical concepts effectively.
PREFERRED QUALIFICATIONS:
- Experience with cloud platforms (e.g., GCP, AWS, or Azure) and distributed computing frameworks.
- Proficiency with version control systems (e.g., Git) and containerization tools (e.g., Docker).
- Familiarity with healthcare data standards and regulatory requirements.
EDUCATION & EXPERIENCE (REQUIRED):
Bachelor’s degree in scientific or analytic field and five years of relevant experience, or a combination of education and relevant experience.
KNOWLEDGE, SKILLS AND ABILITIES (REQUIRED):
- Knowledge of key data structures algorithms, and techniques pertinent to systems that support high volume, velocity, or variety datasets (including data mining, machine learning, NLP, data retrieval).
- Experience with relational, NoSQL, or NewSQL database systems and data modeling, structured and unstructured.
- Experience in parallel and distributed data processing techniques and platforms (MPI, Map/Reduce, Batch).
- Experience in scripting languages and experience in debugging them, experience with high performance/systems languages and techniques.
- Knowledge of benchmark software development and programmable fields/systems, ability to analyze systems and data pipelines and propose solutions that leverage emerging technologies.
- Ability to use and integrate security controls for web applications, mobile platforms, and backend systems.
- Experience deploying reliable data systems and data quality management.
- Ability to research, evaluate, architect, and deploy new tools, frameworks, and patterns to build scalable Big Data platforms.
- Ability to document use cases, solutions and recommendations.
- Demonstrated excellence in written and verbal communication skills.
CERTIFICATIONS & LICENSES:
None
PHYSICAL REQUIREMENTS*:
- Frequently sit, grasp lightly, use fine manipulation and perform desk-based computer tasks, lift, carry, push pull objects that weigh to ten pounds.
- Occasionally sit, use a telephone or write by hand.
- Rarely kneel, crawl, climb, twist, bend, stoop, squat, reach or work above shoulders, sort, file paperwork or parts, operate foot and hand controls.
- - Consistent with its obligations under the law, the University will provide reasonable accommodation to any employee with a disability who requires accommodation to perform the essential functions of his or her job.
Additional Information
- Schedule: Full-time
- Job Code: 4734
- Employee Status: Fixed-Term
- Grade: K
- Requisition ID: 106579
- Work Arrangement : Hybrid Eligible
Read Full Description