Hire an AI product manager in Minneapolis.

Questions from Minneapolis retail and healthcare product teams.

For retail product recommendation AI, what is the evaluation framework. How do you measure whether it is actually increasing revenue?

The evaluation framework for a retail product recommendation AI has two tiers. The first tier is model quality: offline metrics that measure whether the model is surfacing relevant recommendations on a held-out dataset. The standard metrics are precision at K (what fraction of the top K recommendations were purchased or clicked) and recall at K (what fraction of purchased items appeared in the top K recommendations). These are necessary but not sufficient to measure revenue impact. A model with good precision and recall metrics can still fail to increase revenue if the recommendations surface items the customer would have found anyway or items with lower margins than alternatives. The second tier is business impact: an A/B test with revenue as the primary metric. The A/B test design is a product decision. The PM defines the assignment unit (is it the user, the session, or the product page?), the minimum detectable effect (how much revenue lift justifies shipping?), the test duration (how many weeks to achieve statistical power?), and the guardrail metrics (what negative signals trigger an early stop?). Mid-market retailers that skip the A/B test and measure only offline model metrics frequently report strong AI performance while seeing no measurable revenue change.

What UHG-specific data governance requirements should be in a spec before building for UHG members?

Building AI features for use by UnitedHealth Group members or within the UHG ecosystem requires five data governance requirements that go beyond standard HIPAA compliance. First, Business Associate Agreement scope: the spec must identify which data elements flow between the vendor product and UHG systems and confirm the BAA covers the specific AI use case, not just data storage. Second, data minimization: UHG vendor agreements typically require that the vendor collects only the minimum necessary PHI to perform the contracted function. The spec must define what member data the AI feature uses and justify each element. Third, de-identification standard: if the AI model is trained on data derived from UHG member interactions, the spec must define the de-identification method (Safe Harbor or Expert Determination) and confirm it meets HIPAA requirements. Fourth, model output audit trail: the spec must define what logs are maintained for AI outputs that affect member clinical or financial decisions, the retention period, and the access controls. Fifth, incident notification: UHG vendor contracts specify notification timelines for security incidents that are typically shorter than the 60-day HIPAA Breach Notification Rule window. The spec must define the internal escalation path that supports UHG notification requirements.

How do you decide between building a custom demand forecasting model vs. buying a vendor solution?

The build-versus-buy decision for demand forecasting uses four criteria. First, data uniqueness: demand forecasting performance is driven primarily by the quality and specificity of historical demand data. If the retailer has three or more years of SKU-level sales data, seasonal patterns, and promotion history, a custom model can outperform generic vendor models. If the data history is shorter than 18 months or has significant gaps, vendor models that incorporate external signals (weather, regional events, economic indicators) typically outperform custom models built on sparse internal data. Second, SKU volume and velocity: vendor solutions are optimized for high-SKU-count retailers with stable assortments. A $150M Minneapolis retailer with 50,000 SKUs and frequent assortment changes fits this profile. A distributor with 3,000 SKUs and high seasonal variance may need a custom model to capture product-specific patterns. Third, total cost of ownership: a vendor demand forecasting solution at mid-market pricing runs $8,000 to $40,000 per year. A custom model requires a data engineer (approximately $90,000 per year) and ongoing maintenance. The break-even point is rarely in favor of building for a company without an existing data science team. Fourth, time to value: vendor solutions can be integrated in 6–12 weeks. A custom model built from scratch takes 6–12 months to reach production quality.

What does the integration spec look like for connecting a retail AI feature to an existing ERP system?

An integration spec for connecting a retail AI feature to an existing ERP system covers five elements. First, the data contract: the exact fields the AI feature reads from the ERP, the field names in the ERP schema, the update frequency, and the format. Describing this as 'inventory data from the ERP' is not a spec. Naming the specific tables and fields in the ERP API or export format is a spec. Second, the sync mechanism: whether the integration is batch (the ERP exports a file on a schedule that the AI feature ingests) or real-time (the AI feature queries the ERP API on demand). The choice has latency and cost implications that belong in the spec. Third, data quality handling: what the AI feature does when ERP data is stale, missing, or inconsistent. An inventory count from the ERP that has not been updated in 72 hours may produce unreliable AI recommendations. The spec defines the staleness threshold and the fallback behavior. Fourth, error handling and monitoring: how integration failures are detected, who is alerted, and what the user-facing behavior is when the ERP connection is unavailable. Fifth, write-back specification: if the AI feature writes outputs back to the ERP (updating demand forecasts, flagging reorder points), the spec defines the write path, the validation rules, and the conflict resolution procedure when the AI-generated value conflicts with a manually entered value.