The 2-D median filter, one of the oldest and most well-established image-filtering techniques, still sees widespread use throughout computer vision. Despite its relative algorithmic simplicity, accelerating the 2-D median filter via a hardware implementation becomes increasingly challenging as the window size increases, since the resources required grow quartically with the window size. Previous works, in a non-FPGA context, have shown that separately applying several directional median filters to an image, and then taking the median of their results, yields performance that is competitive with, and in some cases even better than, that of a classic 2-D window median. Inspired by these approaches, we propose a novel way of substituting a 2-D median filter on an FPGA with a sequence of directional median filters, in our case arranged as a pipeline, in the pursuit of an FPGA implementation that achieves better scalability and hardware efficiency without sacrificing accuracy. We empirically show that the combination of three particular directional filters, in any order, achieves this, whilst requiring quadratically fewer resources on the FPGA and allowing for much higher throughput.