The towers were constructed to withstand the impact of the largest commercial airliner of the time, a Boeing 707.  It was thought that it would only occur if a pilot lost his/her way in a fog, thus moving slowly and not fully laden with fuel.  Since then, planes became much larger, and they were loaded with fuel and moving at high velocity.
 

The exterior walls actually held up the towers.  Most high-rise construction is "curtain-wall", i.e. the exterior walls don't hold up anything except themselves. This can be observed when such buildings are built, first the steel beams go up, then the floors, the exterior walls go up only to enclose the structure, but don't support anything. The WTC exterior walls were bearing walls, and actually held up the building (along with the center core), allowing each floor to have nearly an acre of open space without columns. The four exterior walls of each building functioned as a huge box beam, giving it great strength.  A building of conventional construction could well have collapsed immediately when hit by a plane, but this didn't happen because of the design.  The weight above the impact area was transferred to adjacent areas, allowing the building to remain standing long enough for occupants below the impact floors to evacuate (occupants above were doomed, since the stairs and elevators were destroyed or blocked by debris, and the roof could not be used by helicopters because of the TV antenna on the north tower).  The exterior walls also served to contain the collapse of the north tower, confined the material within the area of the building; everything went straight down and pancaked the floors. This 'pancaking' also suggests why there were so few survivors of the collapses — out of about 3,000 people dead or missing, there were less than 20 survivors saved from the collapsed buildings, no later than the day after the collapses.
 

Each floor was held up by trusses (zig-zag steel pieces welded to angle iron) connected to the exterior walls. This type of construction is not as resistant to high temperatures than I-beams would have been because of the thinner materials used. It seems likely that the floor-wall connectors failed at the fire floors because of the heat of the fires, significantly weakening the building structure and causing catastrophic failure.  The TV transmitting antenna on the top of Building One wound up standing vertically on top of the rubble, showing that everything went straight down (see photo). The collapses were probably caused by the heat of the fires which weakened the core in the north building on the floors which had been impacted, any sprinkler systems and standpipes were most likely severely damaged by the planes' impacts, as well as not being designed to handle a fire of other than what was in the occupied spaces (as you know, a flammable liquid fire cannot be extinguished with water). The core of the north tower collapsed, and the floors above came down on them, all this was too much for the next floors down, etc.   The south tower collapsed because the fire weakened the connection between the trusses and the exterior walls.  Although the south tower was hit last, it collapsed first because there was more weight above the impact floors. Another site explains this in greater detail - read it here.
 

The towers were not constructed to NYC building or fire standards. Since they were built by the Port Authority of NY and NJ, they didn't have to comply.  Consequently they weren't built with roof tanks or sprinkler systems! As a compromise they were built with double gypsum board walls to slow any fire, which explains the large amount of dust generated when they collapsed.
 

Each of the towers, whose floors were one acre in size, had only three stairways, and each was wide enough to accommodate only two people. (FDNY Battalion chief John Salka, at the Fire Department Instructors Conference in Indianapolis; as reported by Newsday on 4/11/02).  There weren't enough stairways to accommodate all the evacuees on their way down and the firefighters on their way up to fight the fires.  On the other hand, no tall building is designed to evacuate all floor simultaneously because the stairways would take up so much room on the lower floors that there wouldn't be room for anything else.
 

City Code didn't address the potential problems of the towers.  Even though each tower was essentially three buildings stacked above one another (sky lobbies at the 44^th and 78^th floors), deputy fire safety directors were not required at either sky lobby after the installation of an advanced fire alarm system in 1997 (which was reportedly beyond that required by city code).  There was no one in authority to direct occupants above the ground floor (other than through the PA system), nor was there any way for the fire safety director on the ground floor to view the situations at either sky lobby (since TV cameras at each location had been removed). (Newsday 4/11/02).  An authoritative estimate of time required by firefighters to climb stairways with appropriate equipment and hoses is 1 minute per floor.  At that rate the firefighters would have to climb for more than an hour before reaching the fire floors, and probably be exhausted by the time they got there.

The north tower collapsed from the center out.  When the fires sufficiently weakened the remaining core steel, the building collapsed from the center, and the exterior walls followed.

The south tower collapsed from the outside in.  The fires there weakened the floor trusses holding the exterior walls together, and when their connections to the exterior walls failed the building collapsed from the outside in.  Additionally because the plane hit at a lower floor there was more load on the damaged area.

The primary cause of the collapses was the burning of the contents of the buildings.  Structurally the buildings probably would have been repairable, but the burning contents (carpets, furniture, paper, etc.) weakened the structures and caused the collapse.  The aviation fuel started the fires but burned off before causing any structural damage.

More complete explanations:  FEMA Report PBS Report;  
 

While the primary cause was the fires, roof truss failures finally allowed the towers to collapse.  Each plane removed many structural supports on the impact floors, and the loads transferred to the roof trusses.  When the fires weakened the remaining beams on the fire floors, the towers fell.
 

This program also provided recommendations for changes in building codes worldwide in light of the WTC experiences:

Cores should be protected from impacts and fires by concrete.  Both towers' cores were only shielded by double gypsum board walls, which were destroyed by the planes and fires.  If they had been better protected, more people could have escaped from above the fire floors.  The sprinkler system in the core was also destroyed in the core, leaving no water to fight the fires.

Spray-on insulation is not sufficient to protect trusses.  The impacts blew off the insulation, leading to structural failure much earlier than if stronger insulation had been installed.

Stronger connection from exterior to truss would have allowed the buildings to stand longer.  Because the exterior walls were dependent on the trusses, when the connections failed the buildings collapsed.

Public Address systems must be available to advise occupants what to do.  After the planes hit, people were left on their own without knowing whether to evacuate or remain.

There should be some redundancy in the sprinkler/standpipe system.  Because the standpipes in each tower were destroyed by the planes' impacts, no water was available to extinguish the fires.

Perhaps the building contents should be more fire-resistant.  If the contents hadn't burned with such ferocity, perhaps the fires could have been extinguished.

Floor trusses would not be permitted.

All buildings would be required to have evacuation plans.

Stairways would be required to be wider, better protected, and further apart.

Elevator shafts would have to be smoke-proof.

Ventilation systems would have to be designed to limit the spread of smoke.

Structures around lobbies and loading docks would have to be strengthened.

All building fire safety staff would be FDNY certified.