to attack you need to hold LMB, release at any time to stop attacking, be sure to be in combat mode(press "tab" to switch). the character automatically blocks when you are not attacking, how well he blocks depends on your positioning and movement compared to the attackers position and movement.
To get into combat mode, you must press TAB, your cursor will then turn red.
Red cursor=Combat mode
As for blocking, once in combat mode, use your cursor to face your opponent and their attack, you will automatically parry as long as you are not in the middle of a swing.
In Exanima, it is possible to walk by NPC's w/o having to fight them, some are triggered by you being in combat mode, some are triggered by you bumping into them, even if you are not in combat mode and some are triggered by the fact they hated their life and are even more miserable as a zombie ( Just looking at them will trigger them)
Be sure to read through the Quickstart guide (found in the settings menu under the Controls tab), lots of great tips. The game is tough, this isn't going to be your everyday slash and grind. It takes a lot of practice, some have gotten very good, while others (yours truly included) still struggle to advance through the dungeon.
I have found it helpful to grab a shield right off the bat ( lids make great ones) , then in combat, use that to lessen a lot of blows.
Also some attacks only take down secondary health, stamina, (Bright yellow that refills over time) while others take health (red, does not refill over time), getting either yellow or red to 0 will kill you. Some lesser blunt weapons 2x4, etc do secondary health dmg while sledgehammers, metal rods and cutting weapons do primary health dmg. The type of armor and how you wear it will affect those diff dmgs as well.
The learning curve is steep, the combat requires you to actually learn how to use the controls and a lot of practice. But in the end it is really worth it. You may feel like it is unresponsive and the truth is that you are not understanding how it works, but do not get frustrated, just learn and practice.
Everyone here is giving you pretty solid advice. The 2H Longsword in the Arena is probably the easiest weapon to learn with given it's reach and speed. Once you feel more comfortable with basic combat, the Shield and Sword becomes more viable. The Shield and Sword is actually my favorite weapon and the only set I've managed to complete the Expert Arena with.
Practice, practice, practice! It helps a lot to familiarize yourself with LtR swings as well, they are particularly useful against the AI.