How much are you watering the tree? Could be lack of water n pests? Also have you seen any bugs on your tree? This could be causing issues. Stink Bugs suck the sap from young shoots in summer which causes the wilting of young shoots and flower stalks which generally turn black and shrivel.