Here's what GPL software tools you can easily use to create an open source podcast in Windows:
Audacity to record
LAME to convert to mp3 format
Escapepodder to create the RSS feed
Whatever free FTP client to throw it up on a server (can use Windows' own FTP program)
One of the goals of Escapepodder was to have a flash ready application that anyone, anywhere could use to create RSS feeds in a tiny package from a flash drive. There is a flash drive ready version of Audactiy, as well.