I found myself a work arround, this will probably break the ical feed and export for most other ical clients, but since I only use it in google calendar I don't care. It seems to be a bug with google calendar which hasn't been fixed for the last two years.
Anyway, I replaced
return $ical_info;
with
$umlaut = array("ü", "ä", "ö", "Ü", "Ä", "Ö");
$vokal = array("ü", "ä", "ö", "Ü", "Ä", "Ö");
$ical_info_latin = str_replace($umlaut, $vokal, utf8_decode($ical_info));
return $ical_info_latin;
pretty much at the end of CalFormatUtilities.php