The built in podcast app checks on podcasts only when it determines is a good time to. It checks more often when plugged in and on wi-fi. If you want to "pull" a podcast, there are several podcast apps that get good reviews in the store. One that I use to pull a podcast is bringcast. It has a free trial.
As an additional note, with the addition of the new ...
The built-in Windows Phone 8.1 Podcasts app supports this.
Beside the podcast artwork display, there is a grey "1.5x" label. Press this once to select it (it goes white) and playback runs at 1.5x speed; press it again and it switches to 2x speed. Press it once more to deactivate it (switches back to grey "1.5x").
Open podcasts app.
Open in-app settings.
Choose how much episodes you want to keep.
The older ones will be auto deleted.
Currently there is no option to delete podcasts once you have listened, in the built-in podcast app.
In oder to get podcasts into your podcast app you actually have to subscribe to one, either by searching our by entering a feed.
You still can listen to the mp3s though the music app.
It might be easier though to try one of the audio book apps from the store - I think those offer the functionality you're looking for.
To subscribe to a podcast in windows phone 8.1, you either enter the url to the podcast feed, or search inside the app. The app uses Bing to find podcasts, not the store any longer. Currently, these are the only two ways to subscribe to podcasts. The phone then manages the listened to episodes.
OK having spent the last few days looking for the right solution here are my thoughts (for any Zune lovers migrating to WP8 who land on this post).
Firstly I don't think the Windows Phone App for Desktop has anything going for it. It's such a poor replacement for Zune syncing that persisting with it is an exercise in frustration.
Secondly, I should say ...