The focus of the work will be divided amongst these questions:
What are the relevant values when it comes to data-labelling across different industries and sectors? This will help us build the value map, narrow down the list and be able to recommend specific values to projects from specific industries. We can reach out to organizations/ run a survey/ contact external outsourcing data-labelling providers to partner up with them who would run the questions by their clients etc.
How do we build the value-assessment methodology to capture the values that the data-labellers have without them being able to be untruthful or to cheat the assessment? The tools shared above may come in handy. We can also look at already existing work of sociologists & psychologists related to values and what they used in methodologies to determine what values a person has. Once we know what values we are assessing the data-labellers by (from the value map), settle on a methodology (whether it is a speed test, like the Implicit Association Test, on a scenario with questions or anything else that you can propose), we will then be able to start putting the methodology together.
The focus of the work will be divided amongst these questions:
What are the relevant values when it comes to data-labelling across different industries and sectors? This will help us build the value map, narrow down the list and be able to recommend specific values to projects from specific industries. We can reach out to organizations/ run a survey/ contact external outsourcing data-labelling providers to partner up with them who would run the questions by their clients etc.
How do we build the value-assessment methodology to capture the values that the data-labellers have without them being able to be untruthful or to cheat the assessment? The tools shared above may come in handy. We can also look at already existing work of sociologists & psychologists related to values and what they used in methodologies to determine what values a person has. Once we know what values we are assessing the data-labellers by (from the value map), settle on a methodology (whether it is a speed test, like the Implicit Association Test, on a scenario with questions or anything else that you can propose), we will then be able to start putting the methodology together.