Former Wales flyhalf Hayward joined head coach Wayne Pivac's coaching staff in November last year but departs following their disappointing Six Nations campaign in which they lost four of their five matches to finish fifth.
"I have worked alongside Byron for six-and-a-half years, he is a dedicated coach that has had great success in his career," Pivac said in the statement.
"On reflection of the last campaign it was mutually decided that the best way forward for Wales and for Byron is for him to step down from his role.
"In the short-term, for the upcoming four matches, the existing coaching team will manage the defence based on the foundations already put in place."
Wales kick off their Autumn Nations Cup campaign against Ireland at the Aviva Stadium on Friday. They will then play Georgia and England before the finals weekend.
"When I took the role last year I wouldn't have planned to be leaving as the squad prepare to start the new international season, I believe it is the right decision for myself and the squad as they enter a new campaign," Hayward said.
"I would like to wish Wayne and the squad all the best going forward and I'll be fully supporting them."
(Reporting by Shrivathsa Sridhar in Bengaluru; Editing by Michael Perry)