Collaborates with immediate team members on defining metrics and frameworks for evaluating somewhat complex AI/large language model (LLM) systems and agentic product performance. Experiments with model before selection (e.g., LLM, prompting approach) for tasks within a product or solution Diagnose basic issues and identifies opportunities to drive improvements within AI/LLM systems within domain area. Prototypes and develops AI models using containerized environments (e.g., AI Azure Foundry, LangChain) to operationalize, build, and validate scalable tools/applications/solutions for use by the immediate team. Develops and deploys systems for basic retrieval-augmented generation workflows, assisting in the selection of data sources, retrieval systems, and generative models to ensure quality output. Assists senior members in assessing and refining LLM systems and models.
Understands where to acquire data necessary for successful completion of the project plan. Utilizes querying, visualization, and reporting techniques to describe acquired data, including format, quantity, identities, and other surface properties. Explores data for key attributes and contributes to the development of data quality report describing results of the task, initial findings, and impact on the project by leveraging appropriate AI tools (e.g., Codex, GitHub Copilot). Collaborates with others to perform data-science experiments using established methodologies, statistics, optimization, and probability theory for general purpose software and statistical packages. Assesses different tools and techniques and selects the appropriate one. Serves as an effective partner in data preparation efforts to Solution Architects, Consultants, and Data Engineers. Adheres to Microsoft's privacy policy related to collecting data, preparing data, and using AI tools (e.g., Codex, GitHub Copilot). Identifies data integrity problems.
Leverages knowledge of modeling and machine learning solutions (e.g., state-of-the-art statistical analysis, causal modeling, reinforcement learning) to identify the best approach to complete objectives. Understands modeling techniques (e.g., validation, hypothesis testing, optimal use of data) and machine learning in order to select the correct approach to prepare data, train and optimize the model, and evaluate the output for statistical and business significance. Understands the risks of data leakage, the bias/variance tradeoff, methodological limitations, etc. Writes all necessary scripts in the appropriate language: T-SQL, U-SQL, KQL, Python, R, etc. Constructs hypotheses, designs controlled experiments, analyzes results using statistical tests, and communicates findings to business stakeholders. Effectively communicates with diverse audiences on data-quality issues and initiatives. Understands operational considerations of model deployment, such as performance, scalability, monitoring, maintenance, integration into engineering production system, stability. Develops operational models that run at scale through partnership with data engineering teams. Understands and keeps up to date on embedding models and vector databases.
Learns and understands the current state of the industry, including knowledge of tools, techniques, strategies, and processes that can be utilized to improve process efficiency and performance. Maintains knowledge of current trends within the discipline. Attends internal research conferences and participates in on-hands training, when appropriate. Actively contributes to the body of thought leadership and intellectual property (IP) best practices.
Qualifications
Required Qualifications
Doctorate in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or related field
OR Master's Degree in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or related field AND 1+ year(s) data-science experience (e.g., managing structured and unstructured data, applying statistical techniques and reporting results) or consulting experience
OR Bachelor's Degree in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or related field AND 2+ years data-science experience (e.g., managing structured and unstructured data, applying statistical techniques and reporting results)
OR equivalent experience.
Other Requirements
Ability to meet Microsoft, customer and/or government security screening requirements are required for this role. These requirements include but are not limited to the following specialized security screenings:
Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter.
Preferred Qualifications
Doctorate in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or related field AND 1+ year(s) data-science experience (e.g., managing structured and unstructured data, applying statistical techniques and reporting results) OR Master's Degree in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or related field AND 3+ years data-science experience (e.g., managing structured and unstructured data, applying statistical techniques and reporting results) OR Bachelor's Degree in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or related field AND 5+ years data-science experience (e.g., managing structured and unstructured data, applying statistical techniques and reporting results) OR equivalent experience.
Experience with observational causal inference methods, design of controlled experiments, regression modeling, and hypothesis testing.
Experience evaluating LLMs and/or AI agents — designing offline/online evaluations, defining quality metrics, and measuring model behavior.
Experience designing surveys and analyzing response data
Proficiency in Python programming
Proficiency in analytical query languages like SQL, KQL (Kusto Query Language), or similar analytical query languages.
Experience developing analytics on Spark-based big data platform technologies such as Databricks.
Data storytelling — crafting and presenting compelling data stories to business decisionmakers
Familiarity with developer productivity metrics and frameworks (e.g., DORA, SPACE, or similar).
Demonstrated effective use of AI coding assistants in daily data science development workflow
Data Science IC3 - The typical base pay range for this role across the U.S. is USD $102,100 - $202,200 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $133,800 - $219,200 per year.
- Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here:
- https://careers.microsoft.com/us/en/us-corporate-pay
This position will be open for a minimum of 5 days, with applications accepted on an ongoing basis until the position is filled.
Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance with religious accommodations and/or a reasonable accommodation due to a disability during the application process, read more about requesting accommodations.
