It would be best to have a small Knowledge Map and Learning Map about what validity is, with validation evidence for you to explore to see what we mean. We are working on that Knowledge Map and Story for explaining Validity and Validation in the EduMetrics Validity Works Lab. For now we will explain it at a high level.
In more detail, Validity Stories tell how much of the many aspects of validity have been validated and how far along the validation process has gone. Validation manuals are usually readable mainly by experts in the domain of knowledge, and may have literature reviews and research studies in the special language of advanced subject domain – whether medical , statistical, or mathematical, or…. Validation stories, by contrast, are designed to be interesting and understandable to ordinary people as well as to experts. Most end users of a product that make claims about benefits in use are not expert in the technical, scientific basis for providing validation evidence for claims. But the end users are really the ones who in the end validate or invalidate the claims. Think about a medical test, a medication or prescribed treatment, an educational or professional examination. Think about any product claimed to have some special benefits. Problem is, validity and validation have long been a pretty difficult topics to understand by most end users, yet it is the end users who need most to understand what the values and risks are as they follow the advice of doctors or lawyers or teachers, or other professionals who should be up on how sound a validation story actually is for their claims.
In the era of fake news, politically biased groups of fact-checkers do not cover very much of what a strong validation story would include. EduMetrics Institute is trying to make validity and what makes a great validation story accessible, and promote its use broadly. Too many of us, non of whom are expert in the domains where we must trust claims, need to learn to evaluate claims we are counting on the right knowledge under it, clear structure of the knowledge domain, and evidence behind the claims.