SECTION TWO
Feedback From Consultation Phase
2.1 Introduction
The nature of the consultation process allowed stakeholder groups to engage with the process in a variety of ways and at a number of levels. The feedback outlined in this report has been collated from written submissions, information gathered from group discussions at regional meetings as well as issues raised at meetings with stakeholder associations.
2.2 A Quality Standard/ Good Practice Guidelines for YOUTHREACH and Senior Traveller Training Centres
In the exploratory phase stakeholders were asked to outline what they considered to be the key elements of a quality YOUTHREACH and Senior Traveller Training Centre. These elements were initially outlined in the report on the exploratory phase and formed the basis of a draft list of standards that was presented and discussed during centre based and regional consultation meetings. Based on the feedback received further changes have been made to this draft list of standards, the latest version of which is outlined in Appendix 3 of this report.
It was widely acknowledged that the identification and documentation of existing good practice was a valuable exercise. This is especially important when one considers the organic development of the YOUTHREACH programme over the past decade. A great deal of learning and innovation has occurred. The nature of the work in centres has become highly specialised. The programme has developed a considerable level of expertise in providing an alternative and effective education programme for a significant group of learners within the education system. The development of a quality standard will assist centres in acknowledging good practice and identifying areas for improvement. It can also be used as a tool for evaluation.
In general stakeholders were satisfied with the draft list of standards but acknowledged that many require further clarification and refinement. Much of this teasing out will take place in the development phase. It was acknowledged that the draft standards are in essence guidelines for good practice. As guidelines they ought to give a clear indication of the key elements that should be in place but ought also to allow for local flexibility in the way in which standards are achieved. It follows therefore that standards should not be too prescriptive in nature.
It is essential that the quality standards would reflect the nature of the programme. YOUTHREACH is one of the few programmes within our education system that has no minimum entry level. The programme is holistic in nature and based on the very diverse needs of the learners. Unlike mainstream education there is no set curriculum although similar courses and supports are provided. Learning and development occurs at a pace that is appropriate to the trainee. As needs change, programmes must evolve in order to sustain their relevance. It was acknowledged that although Whole School Evaluation will move the focus of inspection in post primary schools from individual teacher performance to the evaluation of the school in its totality, student achievement will still be measured in terms of examination results. Participants felt that within YOUTHREACH and Senior Traveller Training Centres, it would be inappropriate to measure success for learners solely in terms of exam results or certification outcomes. It is widely recognised that much of the success of YOUTHREACH and Senior Traveller Training Centres has more to do with developing life skills and building self-esteem than it has to do with achieving certification. This does not imply that the provision of certified courses is not important for trainees.
Practitioners suggested that assessment of trainee needs would continue to be a key element of programmes and quality would therefore be measured in terms of how a centre responds to that need. This may involve a range of supports as well as the provision of certified and non-certified programmes of education. The focus therefore should be on the service provided for trainees rather than the standards achieved by trainees.
The synthesis process in the development phase was welcomed as an opportunity for all stakeholders to be represented in the decision making process that will lead to the agreement of a quality standard for all centres.
2.3 Centre Based Development Planning
Following the development of good practice guidelines as a quality standard, the challenge facing all stakeholders is to identify a course of action that will embed a quality assurance process into the way in which centres operate. In operational terms how do we ensure that quality assurance stays on the agenda? Bearing in mind the evolving nature of the programme and the changing needs of the learners, an important consideration is the management of change.
Stakeholders identified the process of centre development planning as the most appropriate way of dealing with these challenges. In the exploratory phase the development of centre plans was identified as a key element of a quality programme. During the consultation phase this process was highlighted as a possible key building block within the overall quality framework. If one acknowledges that flexibility should remain an intrinsic aspect of the programme then it follows that in order to achieve this objective, high levels of planning and organisation are required. Practitioners will readily identify with this concept. The provision of an ever -changing, needs-based programme with flexible options and a focus on the learner as an individual can only be achieved by dedicating time to a cyclical process of clarifying procedures, developing policies, reviewing provision, setting objectives and evaluation. The majority of centres currently engage in some if not all of these processes. It is clear however that this is not the case in all centres and some are at different levels of development. A centre development plan will provide an opportunity for the implementation of the good practice guidelines as outlined in the draft standard. The centre plan should contain the centre mission statement, aims and objectives, procedures and policies, review of key areas, recommended improvements and proposed actions. This is not a quick process. Centres may plan to work towards a number of improvements over a period of 3-5 years and may wish to address the more urgent issues first.
The problem encountered by practitioners in relation to this work is the absence of guidelines and support structures. Such resources would greatly assist centres in carrying out this essential work. Stakeholders strongly recommended the establishment of a support structure, the functions of which would include the development of guidelines for centres in relation to centre development planning and the overall quality assurance process as well as the provision of training and advice for stakeholders. The Leaving Certificate Applied Support Service was regularly cited as an appropriate model to replicate. It was also suggested that centre development planning could be supported as part of the School Development Planning Initiative (SDPI). The severe difficulty of implementing a systematic approach to quality assurance unless a support service was in place was generally acknowledged.
Once the support structure is established and has developed the necessary resources, an introductory and training process for stakeholders can begin. Time and resources and ongoing support will be required to assist centres carry out the development of centre plans. Suggestions were made to initiate the process with a pilot centre development planning and internal evaluation phase.
As the changing nature of the programme and the level of flexibility required are greater than those which exist in the mainstream system the amount of time required for centres to carry out this work on an annual basis will also be greater. As a centre development and team building process, it should ideally involve participants, staff, co-ordinators /directors, regional management and boards of directors. In particular, the participation of part-time teachers in this process should be supported. Currently it is generally understood that part-time staff can only be paid for class contact time. Both practitioners and management have strongly argued that the Department of Education and Science should support the implementation of the quality assurance process by clearly acknowledging the need to pay part-time staff for essential non-classroom activity by incorporating planning in their teaching hours, as recommended in the 1996 CHL report Review of Staffing Arrangements for the YOUTHREACH Programme.
2.4 Internal Evaluation
Although centre development planning and the internal evaluation process are interconnected, it is useful to view them as two separate building blocks within the broader quality framework. Stakeholder groups welcomed the proposed development of a structured and systematic process for both internal and external evaluation. Across the country each YOUTHREACH and Senior Traveller Training Centre varies considerably in its experience of evaluation. In some centres a culture of evaluation exists, while in others it is absent. Most centres are positioned somewhere in between.
There appeared to be widespread support among all stakeholder groups for the development of structures to allow for evaluation of centres' compliance with the standard as well as an evaluation of the implementation of recommendations arising from the centre development plans. This may involve centre staff, trainees, boards of management and local V.E.C. management. Again the payment of part-time staff for their participation in evaluation and other quality assurance related activities was highlighted as an issue of concern. It was evident from the feedback that many centres already carry out annual evaluations and appreciate the many benefits of engaging in this process. While the majority of centres had not previously involved local management or trainees in this process, their involvement is now seen as essential. In relation to trainee involvement, it is clear that the Qualification Act requires that procedures be established to provide for evaluation of the programme by learners. Participants strongly welcomed this opportunity and stated a preference for opportunities to evaluate their experience of the programme on a regular basis and not only when they are leaving the programme. Most regularly experience one-to-one participant evaluation meetings with a member of staff, but these evaluations mainly focus on the participant, rather than the service the centre is providing for her/him. Participants welcomed the idea of evaluating centre performance, either using a standard evaluation questionnaire or interviews with the centre staff. A combination of the two methods may provide a more accurate assessment of trainee opinion.
While staff in V.E.C. YOUTHREACH centres strongly supported evaluation of centre performance by trainees, it was generally felt that trainee evaluation should be designed to meet their needs, and should be a separate process to that which would be engaged in by staff and management. In Senior Traveller Training Centres there was much more support for the notion of adult learner representatives sitting around the same table as staff and management in order to carry out an evaluation.
Through the feedback it was clear that some centres were already examining their degree of compliance with the draft standard. Many centres reported general compliance with the draft standard and that the main areas of development remaining were the organisation of related documentation. It is now appears possible that this work may be carried out as part of centre development planning. Other centres reported varying degrees of compliance with the draft standard with some stakeholders reporting that given current budgets and time constraints they would be unable to achieve all standards. The standards commonly causing difficulty in this regard related to premises and equipment, supervision and support for staff, links with the community, public relations and trainee assessment. Generally participants were keen to meet quality standards but felt that inappropriate pay and conditions of employment for staff as well as a lack of systematic training, advisory and technical support and resources such as capital funding caused particular difficulty in achieving such objectives.
For the purpose of internal or external evaluation it is necessary that the centre can produce evidence to demonstrate that standards are met. This can take many forms and may include a range of documentary evidence as well as interviews with staff, trainees and other relevant individuals. At regional consultation meetings participants were asked to propose a range of evidence that might demonstrate that a centre has met each of the draft standards. The feedback on these discussions is outlined in Appendix 3.
Stakeholders voiced a great deal of concern in relation to the additional workload that may be involved in the collation of evidence. Much of the evidence required may be developed or collated under the heading of the centre plan. All centres keep records but there is little consistency in relation to the various systems that are in place. The challenge facing us is to record our work in an efficient and useful manner that would also facilitate a quality assurance process. The development at national level of computer and / or paper based templates for recording information of various kinds was proposed together with guidelines for record control.
2.5 External Evaluation
Although some anxiety exists among stakeholder groups in relation to external evaluation it was generally accepted as a useful and natural follow-on from an internal evaluation process. It also allows for the added advantage of external recognition and affirmation of good practice. External evaluation is viewed as an essential aspect of the quality assurance process. The absence of this key element could possibly undermine the potential effectiveness of the process. A national system of externally evaluating YOUTHREACH and Senior Travelling Centres does not currently exist and therefore a key question remains unanswered. Who will fulfil this role? Out of the many mechanisms and bodies suggested, four predominated:
The manner in which an external body would provide recognition for centres is also unknown at this stage. Receiving recognition through the provision of an award or accreditation is one option. If the inspectorate were involved this may not be necessary, as stakeholders may prefer the application of current inspection procedures and reporting mechanisms.
Whoever fulfils this function it is essential that evaluation should be based on the criteria currently outlined in the draft standard.
2.6 Development Phase
The next phase of this initiative will be the development phase. This will involve the development of a quality standard and principles of an accreditation system for YOUTHREACH Centres, Senior Traveller Training Centres and Community Training Workshops. It will synthesise the recommendations of the consultation process and note areas that are strand specific. It was clear throughout the consultation process that stakeholders wished to remain informed and involved in the development phase. It was specifically recommended that stakeholder representatives would be involved in the synthesis process and that any product of the synthesis process would be made known to stakeholders and opportunities provided for further consultation.
The development phase would also be an opportune time to establish a support service and to pilot a centre development planning and internal evaluation process.
2.7 Participants’ Comments on Quality Standards in YOUTHREACH and Senior Traveller Training Centres
2.7.1 Introduction
During regional consultation meetings participants were asked to reflect on how they themselves, their families and their community view the centres. They were also asked to comment on what makes a centre good or bad and how they might evaluate a centre. Participants expressed appreciation for their involvement in the consultation phase. The following is a summary of the feedback to a series of questions.
1. Are you proud to say you are in a YOUTHREACH / SENIOR TRAVELLER TRAINING CENTRE?
Mixed feelings were expressed in response to this question. On one hand participants were glad to have the opportunity to participate in a supportive education and training programme with certification provided and the possibility of work in completion of the course. Others were very much aware of the negative associations that go with the term "drop outs".
Families are generally happy that the participants attend the programme. Parents of younger participants appreciate opportunities to visit the centre to see what is going on. Older participants often encourage sons / daughters to join the programme.
Community perceptions differ greatly according to the centre. Some centres carry out a lot of outreach work or projects in the community. Participants feel this is very positive and helps the centre to get a good name. Others reported that most people in the community have not heard of their centre and do not know what YOUTHREACH is about and that it should be advertised more. The appearance of the centre was very important. While some trainees felt that their centre was housed in an attractive building others reported that the building looks bad and that this gives a very negative impression to the community.
Comments from trainees in relation to buildings included the following: "cold and dirty", "warm inside, dirty outside", "nice on the outside", "old building", "gone downhill", "bad colour scheme", "clean and warm with good facilities".
The following is a list of comments in response to this question:( not listed in order of importance)
The following is a list of comments in response to this question: ( not listed in order of importance)
In general systematic participant evaluation of the service provided was limited. Many participants are asked if they are happy or if they have any general suggestions to make, rather than being asked to evaluate specific aspects of the programme. The majority reported that time was given to discussing on an individual basis, the development of the individual in the centre. The focus is generally on what the participant is doing rather than the service that is being provided for her/him.
Participants welcomed the possibility of evaluating the programme and commonly welcomed any opportunity where their opinion might be heard. When asked to recommend some ways in which participants might evaluate the provision in the centre the following suggestions were made.
It should ask you what you liked / disliked, what you prepared for next class, did you have any problems, what you learned and the teachers sign at the end.