Because you weren't using fists. You were using giant swords.
Sure, Soulcalibur uses swords, but does it really? Play a round of Soulcalibur and swords are used little differently than any other weapon (or appendage) in a fighting game. You swing them, and if you hit, you take a little bit of damage off.
Barbarian, a 2D fighting game released in 1987 by Palace (and known as Death Sword in the US), used swords properly. Players began a round with "life points", which like any other game in the genre, degrade over the course of a battle with each hit.
But where Barbarian got awesome was that if you timed a swing just right, you wouldn't clash swords or scrape a shoulder. You'd cut your opponent's head off, ending the match instantly (as you can see in the battle below). Brutal, perhaps, and maybe even a little unfair, but that's how swordfights go, so it was great at the time seeing a game actually take that into account.
The instant brutality of a decapitation was only part of Barbarian's appeal, though. It had amazing cover art featuring live models, one of whom would later appear as Wolf on Britain's version of Gladiators, the other a topless model in the UK (and which led to protests in Britain, giving the game some free PR). It was based almost entirely on the works of Robert E Howard (Conan) and Frank Frazetta.
It even boasted pioneering visuals. Rather than simply animating sprites, Barbarian's creator Steve Brown picked up a real sword, went outside, dreamed up 15 individual attacks, copied one more straight from the second Conan movie, then had himself filmed performing them. Palace's artists then copied pixels over stills from the footage, making the game's animation one of the most impressive examples of the decade.
Barbarian was a hit, and from its original C64 version would go on to be ported to just about every home computer system available in the late 1980s. A year later Palace released a sequel, Barbarian II: The Dungeon of Drax, which was nowhere near as good because, while retaining the cover models, ditched the 1v1 fighting for a side-scrolling adventure.