I’d take kids – assuming that they are ok walking, and you have a bit of patience when need rest stops, diversions etc.
From Yung Shue Wan maybe easier, as I think hill climb stretches seem easier, more in bursts (both places at sea level…)
Takes maybe a couple of hours, but add a bit more for the kids.
No steps en route that I recall; but maybe leave the bikes as several uphills where couldn’t cycle (if 5 yr old), and main downhill maybe a bit long and steep for safe cycling by kids.
There’s now fisherman’s village tourist place at Sok Kwu Wan, which I’m yet to add to article; worth a look I’m told.