You can close off a fortress completely in a rather short time. However, that means you'll also forego logging, fruit and plant gathering, and caravans (and fishing and hunting).
While the were curse can spread to dorfs surviving encounters, you ought to brick those dorfs in (separately) over the next full moon to know which one were infected and which ones were not. You can also scour the combat log to see if each units has had blood drawn through a bite, as that's the only way to spread the infection (I've seen different opinions on whether it's possible to resist an infection or not).
As an alternative to closing off the fortress, you can chain up sacrificial animals along the perimeter of the embark: once one of these gets attacked you'll be notified of the presence of a were (note that weres killing visitors does NOT trigger any alert).
Personally I have a recessed and roofed over courtyard with 3-4 access tunnels out from it, with drawbridges at the end of each tunnel. Each tunnel also has a "dogbox" above it with a grate floor overlooking the tunnel, and that dogbox has an animal (typically adopting female cat, as those can't be castrated) pastured in it, and the opening closed with a drawbridge. All entrance tunnels except one have doors in the tunnel, adjacent to the inner drawbridge, so when the guard cat detects a were I order both drawbridges to be raised NOW!, and I lock the doors as the were draws near. The were typically attacks a door, and turns back before the door is destroyed (and if it is, the drawbridge should be closed by that time anyway). The tunnel without doors is the trade tunnel, which has to be wide enough to let wagons through.
Also, when attacked by a were (or a titan), order the closing of entrances immediately, possibly sacrificing the dorfs that are outside at the time (or take the risk to try to save them...).