After Gumi, follow the Mercator-Ryuma road for two or three screens, then turn right into a side path. Make it up to the old guard's house and he'll give you the sword.