{"id":1298,"date":"2020-02-20T04:08:39","date_gmt":"2020-02-19T17:08:39","guid":{"rendered":"https:\/\/blog.datatrue.com\/?p=1298"},"modified":"2024-10-22T18:26:21","modified_gmt":"2024-10-22T18:26:21","slug":"data-qa-why-how-do-it-anyway","status":"publish","type":"post","link":"https:\/\/datatrue.com\/en\/data-qa-why-how-do-it-anyway\/","title":{"rendered":"Data QA is a burden. Here\u2019s why \u2014 and how \u2014 you should do it anyway."},"content":{"rendered":"

[et_pb_section admin_label=”section”]
\n\t\t\t[et_pb_row admin_label=”row”]
\n\t\t\t\t[et_pb_column type=”4_4″][et_pb_text admin_label=”Text”]<\/p>\n

If you\u2019re reading this, you probably don\u2019t need to be convinced that manual QA is a real pain \u2014 it\u2019s slow, it\u2019s tedious, and problems usually slip through the cracks despite your best efforts. Plus, even if you catch and fix every single mistake today, chances are good that tomorrow\u2019s first code push will render all your hard work irrelevant. <\/p>\n

<\/p>\n

<\/p>\n

\"\"<\/figure>\n

<\/p>\n

<\/p>\n

You might also know, or at least sense, that manual QA is a huge time and money sink. Studies show that knowledge workers waste a whopping 50 percent of their time<\/a> \u201chunting for data, identifying and correcting errors, and seeking confirmatory sources for data they do not trust\u201d. If we zero in on data scientists, the picture gets even bleaker \u2014 sixty percent of data scientists<\/a> spend \u201cmost of their time\u201d cleaning and labeling data, and with the majority of those folks earning six-figure salaries<\/a>, that makes for a pretty big QA bill. <\/p>\n

<\/p>\n

<\/p>\n

Unfortunately, though, skimping on data QA is even more expensive. Organizations included in one Gartner<\/a> study estimated that, on average, poor data quality costs them about $15 million per year. Eighty-four percent of CEOs<\/a> also say that they\u2019re \u201cconcerned about the quality of the data they\u2019re basing their decisions on\u201d \u2014 and that uncertainty can lead to lost opportunities, even if the data is ultimately sound. <\/p>\n

<\/p>\n

<\/p>\n

Is it possible to avoid the costs of manual QA without suffering the consequences of bad data? Fortunately, it is. The key is to use manual testing only for a few very specific tasks, and to invest in automation for everything else. <\/p>\n

<\/p>\n

<\/p>\n

When does manual QA make sense?<\/strong><\/p>\n

<\/p>\n

<\/p>\n

If you want to keep your QA costs down, you need to save manual QA for special cases<\/a>. In general, you should only have a human validating data if:<\/p>\n

<\/p>\n

<\/p>\n