Appendix 2: Summary of Study
Evaluating an industry
The starting point for this study was to determine the nature and extent of the “industry”. Even a cursory examination of the AES Melbourne Section website shows just how diverse are the applications of audio, and the skill-sets that practitioners demonstrate. Without a clear understanding of the nature of the industry and the jobs available in it, there is no point in seeking relevant courses.
Since medieval times when education as we now recognise it began, education and training has been about developing and certifying skill-sets of value to the community, and to regulate who was appropriately skilled to undertake trades and professions. The education system has gone through many significant transitions since then as it adapted to the changed industrial and social circumstances over time. The industrial revolution changed the focus from the traditional classical and trades skills to support the then sophisticated emerging industrialised manufacturing industries, and the extraction industries that supported them. The turn of the Twentieth Century saw the beginning of electricity and telecommunications. The First World War brought home the importance of industrialisation and technology. Between the wars the rise of the entertainment industry and the widespread use of telephony, radio and film created an electronics and film industry. The Second World War reinforced the role of technology and the need for a qualified pool of technologists and operators in creating a truly global community. The rise of compulsory secondary education after the Second World War, and the development of a Secondary Technical path for those who didn’t want to undertake a fully academic classical education at secondary level, indicated the increasing complexity that was required to survive in an increasingly complex and technologically rich society. The demise of secondary technical schools and the rise of TAFE, and the upgrading of Colleges of Advanced Education to university status and the rationalising of universities in the 1980s, was an attempt of the system to adapt to the rapidly changing industrial and technological society. Changes to TAFE since then have again been an attempt to further adapt the system in that critical middle-level to the industrial need.
In my many conversations with industry leaders over the years, there has been considerable confusion among them about the significance or relevance to their enterprise of the various levels of qualification. The Australian Qualifications Framework defines the function and outcomes of each of the qualification levels from Certificate 1 to higher degree level. Educationalists of necessity are steeped in this description of the qualifications, but it is difficult to communicate the relevance of a particular course to a specific job, even if such specific jobs are sufficiently numerous and clearly defined as to be the basis for developing such courses. This study attempts to describe the functionality of these levels and relate them to specific skill-sets across the Audio discipline.
But what industry are we chasing with this changing education system? The Australian and New Zealand Standard Classification of Occupations does not directly recognise an Audio Industry – except as part of many other generic industries. Nor does the Australian and New Zealand Standard Industrial Classification define or identify an “Audio” Industry. Instead, the practice of “Audio” appears in many disparate industries, both at the operational and creative level and at the technology design, manufacture and maintenance levels.
A study of education, jobs and skills
The study reported here consisted of three parts: a review of the education context; a review of the industry context; and a survey of industry practitioners to identify common skill-sets and to relate these skills to qualification type and level. The summary report in the links on this page describes the kind of analysis undertaken, and the outcomes of that analysis.
In the first part of this study, I searched the courses in the 2018 Victorian Tertiary Admissions Centre that had a component that could be relevant to the general field of Audio. The choice was guided by consideration of the kinds of occupations that have been described in AES Melbourne Section meetings over the past few years, and from a consideration of the published content of these courses. Although this list of courses is Victorian Centric, a similar range of courses was available in New South Wales, Queensland, Northern Territory and South Australia.
The study also considers the course levels described in the Australian Qualifications Framework, and the expected outcomes specified in that framework. This is an area that seems to be poorly understood by industry members, educationalists and prospective students alike, because there have been significant changes in vocational education in Australia since the Second World War, and particularly in the past twenty years, that has fuelled much of this misunderstanding. Because misunderstanding about the function and specified outcomes of various modern courses was so widespread, the study also briefly reviews the evolution of the education system from its beginnings in craft guilds and early universities, through the changes brought about by changing technologies, to the recent manifestations driven by ideologies and political concerns.
The study also searched key job sites on the Internet to see the range, scope, and education requirements of typical jobs available in the early part of 2018. This list of jobs matched the range and scope of the extant courses and was similarly eclectic, but it better matched the experience from AES presentations than just the courses with “Audio” in the title. Examination of the Australian New Zealand Standard Classification of Occupations revealed that this standard ontology did not well represent the kinds of jobs that are found in the Audio industry. Exploration of the Australian New Zealand Standard Industrial Classification also revealed that there was no such industry as the “Audio Industry”, and the identifiable components of it that were included in the ontology did not well represent the nature of the industry or the jobs that define it.
While there is consistency between the lists of jobs in the marketplace and the relevant courses, the literature is not so clearly defined, leading to sustainment of the confusion among the lay users of the system. A “Practice Analysis” (a list of skills used in the practice of a discipline) derived from a survey of practitioners offered an opportunity to resolve the confusion. This study consisted of two parts: a demographic study to establish the jurisdictional qualification of respondents; and a survey of used and important tasks comprising the practice analysis.
A key finding of the demographic data is that the survey data triangulates and confirms the desk audit data describing the education and industry contexts. Another key finding is that the industry is so diverse and complex that there is no particular education prescription that will suit prospective entrants to the industry: the choice of course of study depends on the individual goals of the interested person.
The practice analysis based on exploring the tasks that practitioners used or considered important returned the study to first principles by exploring how actual practitioners responded to the tasks. Since there were only 47 respondents to this part of the study, a large part of the statistical analysis went to establishing the validity and reliability of those responses in the study context.
Respondents to the survey were asked to indicate on a Likert scale how much they used a particular task in their daily work, and how important a task was even if they didn’t regularly use it. There was no statistically significant difference between the “use” and “importance” variables. These ordinal data were transformed into interval data by a transformation using Rasch/Masters transforms.
The transform process involves using an indicator of respondent ability to estimate the likelihood that a person of a particular ability will have a 50% chance of selecting a particular level on the Likert Scale, and from that, estimating the level of the item as a whole where the 50% chance of selecting that item occurs. This is a measure of the difficulty of respondents selecting that task as part of their practice. Thus, monitor output for quality requires less skill (that is, is “easier”) than capture live sound for TV. The estimation of person ability involves consideration of the totality of responses to the survey, such that if a person performs many tasks but at a low level (selecting 1 or 2) then that person is indicating a broad skill set. On the other hand if the respondent only selects a few tasks as part of their practice, but ranks them at 3 or 4, then that person is indicating a high level of specialist skill in those tasks. The Rasch/Masters calculation that estimates iteratively the person ability and item difficulty then results in a linear measure of the skill required to perform each of these tasks.
The original construct of the survey hypothesised that there are three orthogonal components of skills relevant to the industry: technical; operations; and administration. These classes emerge from the available courses, and from the lists of available jobs. Classifying the responses in this way the three vectors were indeed orthogonal: that is, a score on one dimension did not predict a score on any other in any significant way. On the other hand, while there were many respondents who had a business administration background due to their seniority in the industry, there were few who had relevant production logistics administration which the survey tasks emphasised. A reclassification of the responses into those with a technical background and those with an operations or production background permitted a comparison of the difference in responses between those with a technical background and those with an operational background. It transpired that while the reclassification omitted the administration class of respondents, the remaining reclassified groups had sufficient seniority and experience to be able to reasonably reliably estimate the difficulty of the administration tasks.
Comparing these task lists ranked by skill level apparently required to perform them, one can relate these tasks to courses using the definitions in the Australian Qualifications Framework, and to jobs by considering the requirements in the list of jobs. Thus, the practice analysis based on the tasks confirms the selection of courses and jobs in the literature review part of this study.
The full study explores these issues in some detail. It is available in the links in the left sidebar of the web page hosting this material at:
In addition to the summary of the full study, a link in the left sidebar of the above website provides access to the full study that lists the courses uncovered in the desk audit and a sample of the kinds of jobs that were advertised around the time of the study. If there is a course that should be included in this document, or should not be included, please let me know at the email address at that same website. Persons wanting further details are encouraged to refer to this source.
Dr Rod Staples
September 2018-January 2023