(Sept 5) U18 Evaluation Process Information

If you have not heard about or received an email from SPMHA yet, we are holding an SPMHA Evaluation Town Hall meeting on Wednesday, September 8 at 7:30 PM – please preregister here.  This meeting will disseminate a ton of information about the SPMHA Evaluation Policies and Process, discuss hot-button topics such as Evaluators, Evaluation Games, Team Creation, and new for the 2021-22 season, a formal Grievance Process.

Several questions arise each year from the U18 membership.  One of the most prevalent is how does my player move up and down the “tiers” based on their game evaluations scores.  In U18 we have a unique scenario in terms of a lower number of registrants, players returning from Elite-stream tryouts at any time throughout our process, as well as insufficient numbers to simply play multiple games with players of a broad skill level.  Therefore, we must mix players of varying skill together in some games and do our best to evaluate these players based on where their skill places them against all players in the division.

We have published the Player Movement Matrix to help visualize the evaluation process for the U18 level for the 201-22 season.  In addition, the SPMHA Evaluation Policies and Procedures Manual has been updated in August 2021.

 

Finally, several key Questions and Answers regarding evaluations can be found below:


Q. I noticed the evaluators are looking at their phones an awful lot.  Aren’t they supposed to be evaluating the players on the ice, not messing around with their devices?

A. SPMHA is using an App-based system called TeamGenius to assist in the process of collection and reporting of the evaluation reports, hence why they may seem glued to their phones.  TeamGenius allows an evaluator to watch the game and use a slider to assign a score of between 0.5 and 5, with half increments available.  Comments may be entered on individual players and skill criteria.  An aggregate score is then reported for each player based on the entire group that played in that game.


Q. Why is Little Johnny’s father in amongst the evaluators while Little Johnny is playing in that game?

A. Probably because the Division Director has concluded that Little Johnny’s father is  decent hockey mind, however that does not mean that Little Johnny is getting an advantage.

To avoid Little Johnny’s father from affecting the scores for other players competing for a spot in a tier, Little Johnny’s father would be assigned to evaluate players who are not in the same position as Little Johnny.

(i.e. if Johnny plays forward, Little Johnny’s father will be assigned to evaluate Defencemen in that game.  In fact, most directors will not allow Little Johnny’s father to evaluate any forwards at all throughout the entire evaluation process, even the games in a level above or below where Little Johnny might be currently being evaluated in.  This ensures that Little Johnny’s father does not have the opportunity to purposely sandbag the scores for any other players competing against his little angel at the same position.)


Q. My U15 or U18 player requested to play Checking in their registration.  Why did he end up assigned to a non-checking team?

A. Part of the job of the Divisional Director is to ensure that each player is placed in the group where that player can ultimately have the best chance for success throughout the entire season.  There are also safety concerns that we must take into account for some players who have never played body checking hockey in previous years, or who evaluate at a lower skill level than the players who end up in the Checking groups.  Using skill-based scores from the evaluations enables a director to determine which players will be able to maintain the pace of the top three groups in terms of speed and skill, an important factor when determining the viability of the player to play Body Checking hockey.  The last thing we want to do is place a player into a situation where they could sustain a potentially serious injury by participating in a Checking game that might be above their capacity to play in.


Q. What are the evaluation criteria used to rank my player?  Who decides on them?

A. Depending upon the level of hockey being evaluated, whether there is Body Checking or not involved, and the position (Defence, Forward or Goaltender,) players will be evaluated on a set of 6 or 7 different criteria.  Each criteria has a weight assigned to it, with some criteria (skating ability, game understand/hockey IQ for example) being assigned a heavier weighting than other criteria (such as shooting, passing and scoring).

These criteria are selected by the Evaluation Committee within the Hockey Operations group within SPMHA, based upon the recommendations of Hockey Canada.  The weighting for each criteria is determined by the Evaluation Committee with input from Divisional Directors who are intimately familiar with the idiosyncrasies of their respective divisions.


Q. Why did my player start with one group of kids, move throughout the process and then end up with some, but not all of the same players?

A. Throughout the course of U18 evaluations, we often do not have the numbers to play an entire game with all similarly-skilled players.  We must mix a variety of skills and evaluated scores.  For example, in evaluating which players will end up in tier 5 U18, we play games with a mix of players at approximately the tier 4 and 5 level, and then one with players at approximately the tier 5 and 6 level.  A player who scores low in a “4-5 mixed game” likely will score well in a “5-6 mixed game”.  Ultimately those players will likely end up on a tier 5 team.  The same can be said for a player who ends up in Tier 4 – they may show less well in a tier 3-4 mixed game than in a tier 4-5 game, a good indicator they need to end up in the middle – tier 4.


Q. My player played with a particular player (or set of players) over several season but ended up on a lower tier team than those players – what gives?

A. The best, and worst, part of the evaluation process is that players are given the opportunity to display their skillset on the ice, against a group of like-skilled players.  While your child may have played with certain individuals in past seasons but not this season can be for a wide variety of factors:

    • Some players develop strength, speed or agility at different rates as they grow physically and return year over year with different strengths.  The player that had no strength to battle in the corner suddenly dominates in puck battles and receives much higher scores than in previous years
    • Some players make the effort to improve their hockey skills or overall fitness over the off-season; some put in a tremendous amount of effort and improve their placings dramatically
    • Some players spend a lot of time playing video games over the off-season or otherwise not developing their hockey skills
    • There are times when evaluators pick up on the good, or the bad, parts of a player’s game and that helps or hinders the player’s overall evaluation.  While every evaluator does their level best to see as much of a player’s games, certain shifts can make or break that game’s evaluation score for that player.  This is one reason to encourage your player to make sure EVERY shift is their level best – if a players stays on the ice too long, evaluators may notice that and can negatively score the player on not knowing when to come off the ice, or may see that player give up on a crucial back-check because they are tired, again resulting in a negative score.

Q. My child scored multiple goals in an evaluation game.  Why isn’t he being moved up a tier?

A. Goals are not the only criteria evaluators are looking for.  Yes, a multiple goal game in an evaluation scenario may be due to your child having a high degree of skill.  It could also, however, indicate that they were in the wrong place at the right time, had a period of good fortune, or could also indicate that they were on the ice against a purposeful lesser-skilled line up.


Q. Why does SPMHA not use independent evaluators?  I would happily pay the few bucks extra for the use of independent evaluators who won’t screw my kid over year after year.

A. The SPMHA Executive has debated this topic almost every single year.  The number one complaint we hear from parents surrounds the evaluation process, use pf parent-evaluators and perceived bias against certain players.  There are several reasons independent evaluators are not used:

    • The cost.  First and foremost, the cost for SPMHA to hire a panel of independent evaluators has been estimated several times over the years and it is not just a few bucks that could be absorbed easily by a small fee increase.  Either the players would end up receiving a single evaluation session with a modest increase in registration fees, or the players would receive a sufficient number of evaluation sessions with an increase of approximately $100-250 per player.
    • A single evaluation session may not provide a sufficient picture of a player.  More evaluations (more data points) cost more money.  If a kid had a single bad day, they would not have an opportunity to recover that.  SPMHA takes pride in providing multiple evaluation sessions so players have several opportunities to prove their skill level.
    • In speaking with other associations who utilize independent evaluators, many parents report still being unhappy with there their child ultimately places.  This could be for a number of reasons such as a player having a bad day or being ill/injured during the evaluation sessions.  This does not result in a good return on investment for parents in increased fees.  Other Associations who have used independent evaluators have reported that their top players end up being poached by non-sanctioned leagues (that do not fall under the Hockey Alberta/Hockey Canada membership).  This talent drain is obviously bad for the Association and something SPMHA has consciously tried to avoid.

Leave a Comment