Decision Intelligence News
35.3K views | +2 today
 
Scooped by Decision Intelligence
onto Decision Intelligence News
April 19, 2018 12:04 AM
Scoop.it!

Text Embedding Models Contain Bias. Here's Why That Matters.

Text Embedding Models Contain Bias. Here's Why That Matters. | Decision Intelligence News | Scoop.it
An example of bias in this context is if the incoming message is "Did the engineer finish the project?" and the model scores the response "Yes he did" higher than "Yes she did." These associations are learned from the data used to train the embeddings, and while they reflect the degree to which each gendered response is likely to be the actual response in the training data (and the degree to which there's a gender imbalance in these occupations in the real world), it can be a negative experience for users when the system simply assumes that the engineer is male.
Decision Intelligence's insight:
This is extremely important: just because it's in the training data, doesn't mean it's the best model, especially if you don't want to reinforce problematic patterns of the past. We must take overall multi link systems into account.
No comment yet.
Decision Intelligence News
Decision Intelligence (DI) is the discipline concerned with how organizations and individuals design, support, and improve decision making at scale. It connects actions to outcomes in context by integrating human judgment, data, models, text, and AI into coherent decision systems. Decision Intelligence News curates developments across DI and related fields, including AI, machine learning, governance, enterprise architecture, orchestration, measurement, and decision modeling. Learn more at www.opendi.org and www.learn-di.com.
Curated by Decision Intelligence