Second Study

Drawing on the results of the first study, the second study examined more closely if the respondents have differentiated views on individual aspects of the criteria rated in the first study. Furthermore, the purpose of the second survey was to collect empirical data about the respondents’ evaluation of different visual representations (views) of the same content. It consisted of seven core elements:
  • Introduction (language selection, general survey conditions),
  • Set of questions on the criteria rated as most important for a good online dictionary in the first study,
  • Set of questions on the criteria rated on average as unimportant for a good online dictionary in the first study,
  • Set of questions on different search functions of online dictionaries,
  • Set of questions on different visual representations (views) of the same content,
  • Set of questions on demographics (e.g. sex, age, occupation),
  • Conclusion (thanks, prize draw details).
Using the same methodology as the first study, the second study was designed as an online survey that was conducted both in German and in English. All other general conditions such as the construction of the survey and its distribution were also in accordance with the first study. The survey was activated from August 11th until September 16th. A total of 390 participants completed the survey. The average survey duration was 26.6 minutes (SD = 15.5). 52.3% of the participants selected the English version. 47.3% completed the survey in the German language version.

In the first section, we asked the participants to rate different aspects of the criteria, rated as most important for a good online dictionary in the first study (Reliability of content, Clarity, Up-to-date content, Speed, Accessibility).

Reliability of content: 45.4% of respondents considered the aspect ‘All details represent actual language usage, meaning that all the details provided are validated on a corpus.’ to be most important. 34.4% of the test persons chose ‘All details have been validated by (lexicographical) experts.’ as the most important aspect. Further suggested options included: ‘All details reflect both different types of text and usage across regions.’ (12.1%) and ‘The online dictionary is maintained by a well-known publisher or a well-known institution.’ (8.2%).

Up-to-date content: 41.3% of the respondents selected the aspect ‘Recent linguistic developments (regarding changes in spelling or new typical contexts) are quickly incorporated into the online dictionary.’ to be most important for keeping an online dictionary up-to-date. Over a third (34.4%) of respondents opted for the alternative ‘New words are quickly included in the online dictionary’.  Further suggested options included: ‘Current research is incorporated into the lexicographical work.’ (14.4%) and ‘Edited words are displayed online immediately.’ (10.0%).

Accessibility: Around 33.1% of participants selected the aspect ‘No server failures occur due to maintenance etc.’, whereas 31.8% chose the option ‘The URL / web address is simple and easy to recall.’ to be most important. Further suggested options included:  ‘The online dictionary works properly on different types of device (e.g. mobile / cell phone, PC).’ (19.2%). ‘The URL / web address is simple and easy to recall’ (15.9%).

Clarity: More than one half of the respondents (53.8%) considered the aspect ‘The search window is located in a prominent position, so it is easy to spot.’ to be most important for the clarity of an online dictionary. 25.9% of the test persons chose ‘A quick overview of the most important features and functions of the online dictionary is possible.’. Further suggested options included: ‘You can quickly obtain an overview of the keywords contained in the online dictionary.’ (16.2%). ‘There is an introduction to the online dictionary that is clearly arranged and easy to absorb.’(4.4%).

Speed: When asked which aspect they consider most important for the speed of an online dictionary, 42.1% of the participants selected the option ‘You do not need a password / login to use the online dictionary.’ 29.0% chose ‘Search requests are processed quickly’. The same percentage of respondents opted for the alternative ‘The application works quickly in general; there is hardly any delay when the pages are loaded.’

After that, respondents were presented, both visually and linguistically, with several possible features of online dictionaries in the subsequent two sets of questions focusing on 1.) the use of multimedia and user adaptability (two features that were rated, on average, as partly unimportant or unimportant for a good online dictionary in our first survey) and 2.) the search function. Using a scale from 1 to 7 (where 1 represented ‘Not important / beneficial / helpful at all’ and 7 represented ‘Very important / beneficial / helpful’) participants could rate each presented feature with respect to its importance regarding the use of an online dictionary. To examine possible intergroup differences (i.e. especially related to the participants’ academic and professional background), the answers to the different items were averaged and oriented in the same direction to form reliable scales of importance  with higher values indicating more benefit.

An example will clarify this procedure: Respondents were asked to rate ‘audio pronunciations’ (audio files illustrating the pronunciation of a word, a phrases or a whole sentence). An analysis of variance (ANOVA) shows a statistically significant difference (F = 19.07, p < .00) between respondents who work as translators (M = 5.34; SD = 1.68; N = 146) and those who do not work in that profession (5.97; SD = 1.12; N = 236). A closer look on these findings shows that even more pronounced differences can be obtained if the academic background of the respondents (completed or ongoing academic studies in linguistics) is included in the analysis (cf. Table 1): There is a significant difference (F = 3.16, p < .10) between academic background and the occupational group. Especially translators without a linguistic background rate audio pronunciations on average as unimportant (M = 4.76) compared to respondents with a linguistic background who do not work as translators (M = 6.13).
Table 1: Means and Standard Deviations of the ratings of Audio Pronunciations as a Function of the Academic Background (Linguistic vs. Non-Linguistic) and the Occupational Group (Translators vs. Non-translators). Note that ratings are on 7-point scales with higher values indicating more benefit. .
Background Occupational Group M

(SD)

N
Non-Linguistic Translator
Non-Translator
4,76 (1,71)
5,71 (1,43)
55
86
Linguistic Translator
Non-Translator
5,70 (1,56)
6,13 (0,88)
92
150

 

In another set of questions, respondents were asked to rate four different basic alternative ways of presenting word entries in an online dictionary and to decide which is their favourite. 42.8% of the test persons preferred the Tab-View (cf. Figure 1). The second largest group of respondents (32.8%) chose the Matrix-View (cf. Figure 2).  17.7% of the test persons selected the Explorer-View (cf. Figure 3). Only 6.7% of the respondents picked the classical Print-View (cf. Figure 4).

Figure 1: Tab-View
Figure 2: Matrix-View
Figure 3: Explorer-View
Figure 4: Print-View
Further planned surveys directly address (potential) users of monolingual German online dictionaries, such as elexiko .

Drawing on the example of elexiko, the aim of further studies is to investigate the actual behaviour of online dictionary users (i. a. in predetermined usage situations), how users comprehend the terminology of the user interface, and how users cope with new methods of access, breaking the grip of the alphabet.

By the means of eye tracking experiments, the use and evaluation of search functions and navigation paths will mainly be investigated on the basis of OWID, the Online Vocabulary Information System of German, created, run, and maintained by the IDS with elexiko as one component.