The number of people taking up refereeing has been in serious decline for many years, with many league's complaining of being on the brink of collapse.
And Casey, who is also a member of the FAI Senior Council, believes that official's reluctance to tackle the problem is having a knock on effect throughout Irish schoolboy and amateur football.
"League's are going to have to start taking the refereeing problem seriously," warned Casey. "One of the main, and most obvious, problems is the amount of abuse referees take from the sidelines. But I honestly believe that League officials are not doing enough.
"They have to get out on the sidelines and watch games, listen to the comments and then take action. Otherwise there is no point."
At one point the shortage of refereeing in the Kildare League was so extensive that members of the league's committee were considering reducing the games in the lower divisions to once every two to three weeks.