Since read.shp in the fastshp package returns the polygon data in the form of a list of lists, it is then a matter of reducing it to a single dataframe required for plotting in ggplot2.

library(fastshp)
library(ggplot2)

setwd(system.file("shapes", package="maptools"))

shp <- read.shp("columbus.shp", format="polygon")
shp.list <- sapply(shp, FUN = function(x) do.call(cbind, x[c("id","x","y")]))
shp.df <- as.data.frame(do.call(rbind, shp.list))
shp.gg <- ggplot(shp.df, aes(x = x, y=y, group = id))+geom_polygon()

EDIT: Based on @otsaw's comment regarding polygon holes, the following solution requires a couple of more steps but ensures that the holes are plotted last. It takes advantage that shp.df$hole is logical and polygons with hole==TRUE will be plotted last.

shp.list <- sapply(shp, FUN = function(x) Polygon(cbind(lon = x$x, lat = x$y)))
shp.poly <- Polygons(shp.list, "area")
shp.df <- fortify(shp.poly, region = "area")
shp.gg <- ggplot(shp.df, aes(x = long, y=lat, group = piece, order = hole))+geom_polygon()