AFA Certification & Testing

American Farrier's Journal | 2006

Evaluating Farrier Certification Testing

As Chair of the AFA Education Committee my committee and I have been given the task of developing a plan to increase the pass rate of the AFA Certification testing. This is a standing mandate for the Education Committee.

The AFA Certification pass rate is, anecdotally, 18%. The big question is why. Testers will tell you that those that present themselves for testing are not prepared. Applicants to the test will tell you that the test is arbitrary and geared toward 'contest' shoeing.

My comments are not meant to offend. I am not pointing a finger at any individual or group. I am not condemning the Certification program or those that have worked hard at forming and implementing the plan. This is an attempt to look at a problem, pass rate, and insert a different perspective.

In designing a testing program you must be careful that a test is not measuring the skills and knowledge that one would like to see but rather the knowledge and skills that the participants posses.

For example, fifty farriers present themselves for testing at the Certified Level with all fifty farriers shoeing full time for the past five years. They all support their families, make their mortgage payments and pay their bills with shoeing income. Since their sole source of income is shoeing they are considered professional farriers.

The AFA's "Farrier Certification Program guide" states that; "The Certified Farrier component is available to farriers who have at least one year of horseshoeing experience, and have demonstrated knowledge and skill to perform hoof care on a professional basis."

If only nine (18%) pass the Certified test then the test is not a valid measurement of skills and knowledge. The other 41 obviously have skills and knowledge or they would not be able to support their families shoeing for the previous five years. To the horse owner the 41 farrier's that failed performed work that was detrimental to their horse's well being, that's why they failed.

The conclusion that can be drawn is either (1) the test is invalid in that it doesn't measure the skills and knowledge of the everyday farrier, or (2) that the everyday farrier does not posses the appropriate skills necessary to pass the test. Either way the test is an invalid measurement.

Please note that the term invalid is not a negative term condemning the certification process. It is a descriptive term used to gauge the relationship of teaching to testing. Invalid just says that there is a problem in the relationship. That what is taught and what is tested are out of alignment. There is no blame assigned in this article.

A well -grounded assessment plan has three components

  1. a statement of educational goals
  2. a valid set of instruments to measure achievement of these goals
  3. a plan for utilizing the results from the assessment to form polices to improve the educational process.

1. A statement of educational goals. The AFA has no program to teach horseshoeing. In fact the Mission Statement of the AFA is: "To further the professional development of farriers, to provide leadership and resources for the benefit of the farrier industry, and to improve the welfare of the horse through continuing farrier education." There is no provision for training the layman to be a farrier.

Since there has historically been an adversarial relationship between schools and the AFA, there is no continuity in the training that leads to testing. The schools and the Certification Committee have not united in a clear and concise statement of educational goals. The Certification testing must be a measure of skills and knowledge taught in schools and schools must teach to a standard that gives the students the highest possibility of passing the tests. To increase the pass rate, the exam process has to be the result of agreements between schools and the AFA.

2. A valid set of instruments to measure achievement of these goals. This is the certification test. However, an 82% fail rate speaks to the validity of the test. By definition of the high fail rate, the current certification process is not a valid instrument to measure skills and knowledge.

The current Certification process is an attempt to educate through testing. The AFA is attempting to use the Certification process (an instrument to measure achievement) as a statement of educational goals. This means that the participant must fail the exam enough times to learn the skills necessary to pass. Without a statement of educational goals, a method to teach what will be tested and valid instruments to measure achievement the testing becomes more of a contest than an educational instrument.

3. A plan for utilizing the results from the assessment to form polices to improve the educational process. There is no statistical data maintained for the AFA Certification process. Without detailed data guiding the writing and rewriting of tests the task becomes meaningless. Each certification committee rewrites are nothing more than the reinvention of the wheel.

Without detailed data about scores for each category in the practical exam, each question in all written exams or detailed data about the shoes there can be no plan for education to increase needed skills.

Detailed data must be gathered and maintained so that (1) the areas that show low skills and knowledge can be identified and a comprehensive program developed by the schools and the AFA to teach those skills or knowledge, (2) poor questions or tasks can be identified and clarified or removed from the testing, (3) those testers with high pass or high fail rates can be educated to insure that they are testing for the statement of educational goals established by the AFA and the schools, (4) to identify schools that have low pass rates so that their curriculum may be modified to reflect the statement of educational goals.

It is imperative that the identity of testers and schools in the data gathering be held in strict confidence to avoid negative stigmas that may have personal or financial implications.

The first challenge then is the expression of the outcome in terms of components that are sufficiently complete and detailed for use in broad instructional settings. What does one need to know at which level and how do we incorporate that into school teaching curriculums and AFA testing.

There must be a designation of categories; such as Classroom, Forge and Trimming/Shoeing. Each category must be broken down into sub categories. For instance the Category of Classroom can be broken down into (1) anatomy, (2) conformation, (3) lameness, (4) gaits, (5) shoeing for disciplines, etc.

Each subcategory can then be tested in four areas; (1) knowledge, (2) comprehension, (3) application, and (4) analysis. Knowledge would be remembering previously learned information, memorization. Comprehension would be the ability to grasp the meaning of the information. Application would be applying the knowledge to an actual situation. Analysis would be breaking down objects or ideas into simpler parts and discussing how the parts relate and function.

A simple example might be a rocker toe shoe. At the entry level one might require that the participant identify a rocker toe shoe (knowledge). That the rocker toe shoe enhances breakover (comprehension). Make a rocker toe shoe from a keg shoe (application), and that the rocker toe shoe can be used for shoeing a horse with navciular disease (analysis).

At a higher-level, knowledge may require that the participant know a variety of rocker toe shoes, half round, EDSS shoes, etc. Comprehension may require knowledge of where breakover would be for the various types of rocker toe shoes. Application may be the participant hand making a rocker toe shoe and fitting it to a foot. Analyzing may be an essay discussion of the use of rocker toe shoes for a specific lameness and specifically why the rocker toe shoe works for that particular lameness.

In addition to the four areas there needs to be a chronology in the learning and testing. For example, take the knowledge of horse bones. What does a participant need to know about bones at the Entry Level? In addition to the Entry Level knowledge about bones what else is required for the Certified Level. The Tradesman level must know the Entry Level and Certified Level knowledge plus additional knowledge. No question or skill should be inserted into the testing until the chronology is measured throughout the different levels of Certification. The different test levels cannot be written and designed in a vacuum. Without this building block of teaching and testing the entire process become haphazard.

However, the development of an educational plan can not be accomplished until: (1) schools, the AFA Education Committee and the AFA Certification Committee develop a working relationship, (2) there is continuity between the specific skills and knowledge that is taught and tested, (3) a plan for the collection of statistical data is developed and implemented, and (4) quite bluntly a re-education of those individuals, in authority positions who (a) feel that minor cosmetic errors, that are not detrimental to the horse's well being, are considered a failure and (b) think that the concept of teaching what is being tested is somehow 'dumbing' down the process.

Every University, College or vocational school that teaches an occupation that leads to licensure develops a relationship with those that test for the license. Schools are expected to teach the skills and knowledge that the students are required to know to be licensed. The testing agency is expected to test on the skills and knowledge that was taught.

The licensing agencies and the schools are constantly meeting to discuss problems, current trends and prevailing thoughts in the occupation as well as methods to increase the pass rate. The licensing agencies have clinics, seminars and newsletters in the attempt to work with the schools to increase the pass rate. Accurate and detailed statistics are kept and studied by both the educators and the testers so that problems areas can be flagged and corrected.

To be given broad ideas about what is expected and to then be tested in the abstract is not a valid measurement of skills or knowledge. As long as there is an 82% fail rate, with four to six percent farrier participation, the AFA certification process is not a valid measurement of the skills possessed by the everyday professional farrier.

The necessity of building bridges with educational institutions is immeasurable to the AFA Certification process. The AFA and schools need to develop classroom lectures, forge practices and practical shoeing guidelines that are geared toward teaching the skills necessary to successfully perform on the test. That is education.

Bob Smith, B.A., CF, AFA #504
Pacific Coast Horseshoeing School, Inc.
Chairman, AFA Education Committee
Vice Chair, American Farriers Education Counsel (AFEC)