Neither player could hold serve during the first set, with Ivanovic eventually claiming it on a tie break with the help of Williams' unforced errors.
Williams, 27, raced into a 3-0 lead in the second set only to be broken for a fourth and fifth time. Fourth-seeded Ivanovic, who lost to Williams at last year's Wimbledon and French Open, saved two break points before closing out the victory in one hour, 46 minutes.
Ivanovic, 20, will face Slovak Daniela Hantuchova in the semifinals.
Serbian Novak Djokovic, 20, demolished fifth seed David Ferrer to clinch his fourth consecutive grand slam semi-final appearance at the Australian Open today.
The third seed defeated Ferrer 6-0, 6-3, 7-5 in two hours and 13 minutes.
He will now play with top seed Roger Federer in the semi-finals on Friday.
Three of the eight men's and women's players in the semifinals at the Australian Open are from Serbia: Ana Ivanovic, Novak Djokovic and Jelena Jankovic,
who ousted Serena Williams a day earlier.