Milner was optioned to Triple-A Lehigh Valley on Saturday.
The southpaw struggled Thursday, allowing three runs in one-third of an inning. His strikeout rate (7.7 K/9) and walk rate (5.8 BB/9) also left much to be desired, so he'll head back to the minors. A corresponding move will be announced Sunday.