In a medium-sized mixing bowl or bowl of a stand mixer, mix together sugar, cake flour, baking powder, baking soda, and salt.
Add peanut butter and mix until combined.
Add eggs and peanut oil and mix until just combined.
Slowly add milk and mix on low speed until just combined. The batter will be liquidy. (Don't worry, you didn't do anything wrong. It's supposed to be that way.)
Fill cupcake liners 2/3 full.
Bake for 14 minutes and then test to see if they are done. They are done when a toothpick comes out without wet batter stuck to it. If they are not done, test again in two minutes. If they are still not done, test again in another two minutes.
When the cupcakes are done, remove them immediately from the tins and leave them on a cooling rack (or just on your counter if you don't own a cooling rack) to cool.
Frosting Instructions
Beat butter on high speed with an electric mixer for about three minutes until light and airy.
Add peanut butter and salt and continue to beat for another minute.
Mix in powdered sugar a little bit at a time.
If desired, add more powdered sugar to make the buttercream frosting stiffer.
Notes
Use smooth peanut butter. Chunks in your cupcakes keep the texture from being uniform, and chunky frosting doesn't pipe well and won't look as pretty. If you use natural peanut butter, be sure to mix really well before adding; it's important that the oils are thoroughly mixed in.
I recommend using unsalted peanut butter and adding the amount of salt listed in my recipe. However, if your peanut butter has salt added to it, don't add salt to the frosting without tasting first. It may already be salty enough.
You will need to stop and scrape down the side of the bowl every minute or so while you are mixing. The easiest way to do this is with a silicone spatula. One way to avoid scraping down the bowl is to use a mixer that has a scraping paddle. Many mixers now come with scraping paddles, but you may be able to purchase one if yours doesn't (e.g. the beater blade).
If you plan to make a very large swirl of frosting, double the frosting recipe.
While there are many frosting recipes that you can mix by hand, the best buttercream frostings require lots of air to be incorporated into the butter. To really get this job done right, I highly recommend that you use an electric mixer. A hand mixer or a stand mixer will both work well.
Sift the powdered sugar if needed. Take a look at the actual sugar to see if it has clumps (some brands are clumpier than others). Clumps of powdered sugar will not get integrated into the frosting and may end up getting stuck in your piping tip. Measure before sifting.
The frosted cupcakes last for 3 days at room temperature, up to a week at the fridge, or months in the freezer.