You could always build your pit. Then build the cage in the center of your pit. Attach the cage to a lever outside of your pit. Seal your pit/wall it off. Then use the lever to deconstruct the cage and release the Colossus into your pit.
Then the Colossus would be hostile to everyone and be able to move around the entire pit and attack anyone/anything that falls into it.
That was plan A, but I realized that wouldn't give me any method for retrieving goodies from the smears of goblin paste or rescuing citizens who blunder into the hole accidentally, at least not without dealing with the colossus.
It wouldn't save the citizens who blunder into the pit, but you could simply seal the pit with a single-tile wide raisable bridge and a several tile long single-tile hallway filled with cage traps. Then just re-cage him whenever you want to clean out the pit.
I know it might be simpler to have him chained, but if you are experiencing problems getting him to aggro goblins or have dwarves shoot at him, it could work as a backup.