Here's what GPL software tools you can easily use to create an open source podcast in Windows:

Audacity to record

LAME to convert to mp3 format

Escapepodder to create the RSS feed

Whatever free FTP client to throw it up on a server (can use Windows' own FTP program)

One of the goals of Escapepodder was to have a flash ready application that anyone, anywhere could use to create RSS feeds in a tiny package from a flash drive. There is a flash drive ready version of Audactiy, as well.