Well now this is all very interesting. While Joyce Olivia is busy spinning to reduce the incident the Brit CD is running loose mouthed with inappropriate tweets or whatever. It appears there is more than one design flaw involved here, one in management, one in the ships. Note ships because the same problem must exist in all the ships of that class (Conquest) which are in several of the Carnival Group’s individual Cruise Lines. One would suspect the incident and damage is considerably more than Ms. Olivia has announced. Thanks to that Brit.CD all hands now know what ALPHA TEAM means, still true that “Loose lips sink ships”.
No worries, finally retirement is enjoyable!!!
Good Watch.

Merci M. Clermont, clearly the Press Release from Carnival Cruise Lines cannot be correct. However good luck with finding out the full and true story. Perhaps an NTSB report is our best bet to learn from this incident.
Good Watch.

In the Costa Victoria brochure they also talk about availability designed ” any disturbance in the machinery will cause least possible effect on ship’s speed and passenger comfort…” guess not!

Recalling the original report the word was that the initial fire was extinguished. After some time period there was a ‘reflash’. Now is it possible that human intervention caused that ‘reflash’ by someone in the engine rooms closing that breaker between the two switchboards, which should have dropped out of the circuit as designed, right. In turn this resulted in an overload and complete collapse of the electrical power system. Being a Master and not an Electrical engineer this may sound funny to you guys – still is that a possibility?
Good Watch.

The tie breaker can also be only a non-auto switch it will not trip
What are the qualifications of the engine room crew you may have only 1 eng. with non qualified assistant. what is the required number of gen in std operation. I had seen 4 gen of 6 required for full propulsion with the tie closed at all time .
When you have a fault on one of the gen, the assigned breaker only should trip; what is the total interrupting capacity of the system

The damage to the Switchboard was due to the high temperature fire raging for a few hours. The fire melted the deck between the engine room and the switchboard on the deck above. With the main switchboard and most wiring severly damaged from fire, electricity could not be transfered anywhere. If there was any failure or breakdown
(other than the original engine) it was with the fire extinquishing system in the engine room. This fire should have been out in minutes, not hours.

Good day
I had a theory to propose
The mains switchboard are located above the gen; if there is a fire in one of the gen. and the carbon smoke was able to contaminate inside main swbd.(do not forget it is a 10 KV system) then arc will flash all over the place including the electronic control system. As i was reading on the power plan layout page 4 it appears to me that the 2 swbd. are always connected.Is there a real manual back up control system ;if not and the computer system is flashed out of commission ;no generator can run without gen. control, load sharing reverse power relay synchrocheck relay and so on .

Is it possible there are no emergency generator

MARK: Closely held; remember this is a Cruise Line if anything comes out from an employee they are immediately ‘blacklisted’ and it sure is not going to be released!!
Good Watch.

here ya’ go….

Chinese EMP Attack prompts US Missile Strike After Cruise Ship Crippled

http://www.whatdoesitmean.com/index1421.htm

….as per my opinion ships built in Italy at Fincatieri shipbuildings have not enough mechanical/electrical redundancies…and this fire is a very simple example…agaisnt the one built in KMY (Finland) where the concept of redundancy is very well implemented (very similar to military constructions), the switchboards are two and not one, far away from each other dedicated for each engine rooms and linked by bus bar with tie breakers, the Fincantieri standard is a single room where all is fitted in : 11Kv or 6,6 Kv and low voltages!!!!!! Bad design bad concept

Barry, read the November 15 comment from Rick that said:

“There is an interesting brochure by SAM Electronics – Diesel Electric Propulsion System for Cruise Liner “Costa Victoria” The Carnival Splendor was based on a Costa design and has the same six engine lay-out in fore and aft engine rooms as the Costa Victoria, though the Splendor is a larger and newer ship. The diagrams show a breaker between the two switchboards.”

The diagram shows that the two switchboards are together in the same compartment.

In the Carnival Splendor CD’s blog, John Heald reports what was happening on the bridge about five hours after the incident happened.

“The Guest Services Desk had a line of people saying that their lights were not working and their toilets not flushing. So I went to the Captain who was on the phone still with the command centre and asked him to interrupt the conversation so we could have one that went like this:

JOHN: CAPTAIN, I AM GOING TO HAVE TO SAY SOMETHING ABOUT THE SERVICES ONBOARD, WE HAVE NO ELEVATORS, NO LIGHTS AND NO POWER AND MOST IMPORTANTLY FOR NOW…………THE TOILETS ARE NOT FLUSHING.
The Captain looked at me with tired blood shot eyes. Claudio is a handsome man and being an Italian man means he has won the lottery of life. Yet as I stood there looking at him I saw a man who was aging by the second. Obviously being fat and British still made him a thousand times better looking than me though.
CAPTAIN: LET ME SPEAK TO THE OFFICE, HOLD ON.
The Captain then related my concerns and asked for permission to do this which of course he got. I will say that even if the beards had said “no, wait and see what happens” I would have respectfully ignored them and done it anyway. I had promised from the start of this that I would tell the truth and I wasn’t going to stop now. But my colleagues in the command center agreed with me but the Captain wanted to do one more thing before I spoke to the guests.

“CAPTAIN: JOHN, LET ME CHECK WITH THE CHIEF ENGINEER
The Captain then called on the Walkie Talkie to Chief Engineer Mario.
CAPTAIN: WHATS THE STORY WITH THE POWER AND THE TOILETS ETC
There was silence for a moment and then a voice that sounded like he was about to tell someone that a family member had been killed said “Captain, the main switchboard is finished………………. he paused…………… and then his voice cracking with emotion he said words I will never forget. I cannot give you anything………. nothing…………. no engines, no power…………… everything is finished………… finished…………. finished.”
I will never ever forget the fact that he said the word “finished” three times and each time he said his voice reached new levels of despair. This was his engine room and now it was a dead place, devoid of life and he was taking this very personally indeed.”

Whatever happened, the result of the fire in the aft engine room was that both switchboards were disabled/”finished”, that is rendered inoperative.

Obviously, this should not have happened even after mechanical/eletrical problems in one of engine rooms and even if the problem was compounded by human error.

I am not an expert or an EE. But I’ve worked at a hydro plant for 25 years and kind of know their systems.
I wonder if what we are looking at a master panel in the control room with no separate back up system in another compartment. Take out that one master panel and “poof”, no way to control the generators. As quoted from the CD, “Captain, the main switchboard is finished…”

I just looked at the SAM Electronics’s Costa Victoria brochure. I interpreted it as showing two 10KV buses, in separate compartments, each above its associated diesel generator room.

At the bottom of page 2 the third bullet in the table states “2 Switchboards 10KV…”. Page 3 goes on to state that the main switchboard is located above the diesel generators. The picture at the bottom of page seems to support this.

From this picture at the bottom of page 3, it appears both synchroconverters are in the same compartment, aft of the aft generator room and above the propulsion motor room. However, this should not comprise the ability to supply hotel and ship service power.

As expected, there is a tie between the two switchboards (buses) with two breaker isolation. Ideally, the two buses (aft and forward) should not be electrically tied together but with two tie breakers in series a double failure would have to occur to cause a fault on one bus from compromising the other bus.

I am amazed how this event has completely fallen off the public’s radar screen. The safety of almost 5000 souls was compromised, Carnival is not putting out any information (because they have something to hide IMO), and no one is clamoring to find out what happened.

If I interpret the Costa Victoria Brochure correctly, I do not think the loss of all power was the result of bad design. So what did cause the problem? I assume there was some initiating event that caused an electrical fault in one of the two buses. Somehow, this fault seemed to propagate to the other bus. The buses are in adjacent compartments so did they not maintain proper compartmentalization (open door, inoperable fire barriers, inoperable ventilation isolation systems, etc.) between the two? Were fire safety systems (suppression, barriers, detection, alarms, ventilation isolation, etc.) inoperable prior to sailing? There was a statement about a catastrophic mechanical failure of one of the diesel engines. Did missiles from this failure penetrate the switchgear above it starting an electrical fire? Did the fire then spreads for any of the reasons above?

In my opinion the loss of all power, based on my interpretation of the design from the Costa Victoria brochure, was a preventable event. There was some lapse of something that resulted in loss of containment from the aft engine space to the forward space.

I have to emphasize I have no other knowledge of the event other than what everyone else here has so all my opinions are just that, opinions; and we know the old saying that says something about everyone has an opinion and everyone also has …. .

Mr Clermont
I worked at a shipyard for 10 year as a test engineer. My job wss to startup and test all systems prior to sea trials. One of the tests was testing the reverse power relays. I never had one fail to work. From what I understand if the relay fails to trip, the online generator will try to drive the engine as a motor. It is difficult to even imagine something like this happening, but what would happen if it did?

Thank you for anserwain my question. I’m not sure about computer driven engines.
The reason I asked about reverse power relays is I read about the President of Carnival Cruise Line saying the problem was a “cracked crankcase” and I read on another blog the engineroom crew had worked on the engine and was trying to put it in parallel with the on line generators and couldn’t get the automation to work so they tried to do it manually and the bus tie closed when the generator was out of phase.

Let me offer some thoughts about some of the new comments above:

A typical DG governor, in years gone by, was solely an elctro-hydraulic unit that directly controlled the racks (the mechanical linkage controlling the injectors). The unit developed hydraulic pressure directly by a small internal pump driven by the diesel itself. This hydraulic pressure not only provided the energy to drive the racks open or closed, but the pressure was also the speed signal as it is proportional to rpm. An external electrical signal was used to adjust the speed/load setpoint to which it tried to control for a given pressure. A more up to date arrangement still uses the same or very similar hydraulic unit but also uses a digital based electronic system to tell it what to control to. The electronic device uses electronic speed sensors (in lieu of hydraulic pressure)and, for many reasons, provides for much more precise speed/load control. The hydraulic unit also acts as a self contained backup if the electronics fail. I do not have experience with new state of the art engine systems but I suspect they use a similar arrangement; off engine digital electronic brains and an on engines actuator (probably hydraulic) that implements the digital decision making and acts as a backup if the electronics fail.

With respect to synchronizing (paralleling to an already running system), reverse power relays cannot prevent the breaker from closing out of phase. Reverse power relays are only intended to prevent “motoring” the machine and the breaker must be already closed if this happens. This can occur if, for some reason, the fuel oil supply is lost to the engine so the generator becomes a motor driving the engine.

Catastrophic mechanical damage can occur if the breaker closes substantially out of phase. However, there are relays, typically called “synchronizing relays”, that are designed to assure a safe paralleling experience. Synchronizing is done either manually or automatically. Generally, automatic is considered the safest. Even when done manually, there always a “synch check relay” of some kind assuring the operator cannot screw up too badly.

Now some thoughts about the real world. Synchronizing devices are designed conservatively because the effects of synching out of phase can be so catastrophic. In my experience, it is not a rare event to see synch devices unnecessarily prevent closing the breaker because they are so sensitive. As a result, operators will simply manually synch without the protection of the overly sensitive relays. Although this usually does not result in a problem, intentional protective design features are not used and this is a setup for human error. It is also a scenario that should never be tolerated in any plant that has a critical function.

I cannot imagine a state of the art modern plant, like aboard the Splendor, did not have a fully automatic computer controlled system for synchronizing the machines. However, many minor issues, can prevent the computer from working and might even make the computer think the phase is 180 degrees from where it actually is (e.g. inadvertently rolled potential leads). There should be redundant synch check functions to make sure the computer does make a wrong decision but the backup system might use the same sensors or be manually bypassed because it is over sensitive and prevented synching in the past.

I certainly do not know what caused the engine failure or even if the engine failure was primary or secondary to the event. I do know, without doubt, if the event was caused by an attempt to parallel a machine that went terribly wrong, it went wrong, not because a reverse power relay did not work properly, but because of human error.

When a circuit breakers is not rated to interrupt the available fault current, the result is typically an arc that cannot be extinguished, resulting in a lot of melted and charred metal, sometimes accompanied by an explosion as the rapidly heated air explosively expands. I have seen the result of this type of failure and it is as ugly as it gets.

Synching an “incoming generator” to a “running” system, is a very common event. Utilities regularly synch large machines to what is considered an “infinite” bus. As long as the breaker is closed with a very close to zero degree phase angle, this represents no problem. Just before closing the breaker, the operator will set the rpm to be slightly higher than system frequency (“slow in the fast direction”). This causes the phase angle to slowly drift into and then out of phase. When the phase angle is just a few degrees away from approaching zero, he will close the breaker. Therefore, when the breaker is closed, the rpm is forced to instantly drop to system frequency. The small amount of fuel oil or steam, that was causing the rpm to be slightly higher than system frequency, now drives electrical energy out of the machine. In other words, the generator picks up a little electrical load the instant the breaker is closed so it does not motorize. The operator then bumps the throttle up a little to assure the machine is stable with some small but reasonable electrical load on it. I have synchronized generators driven by 1,000,000 HP steam turbines and it is an easy task, but one that must be done carefully as a good operator never ever depends on automatic systems to mitigate a mistake.

If the Splendor event was caused by a synchronizing error, it was certainly the result of many failures, not just a single problem. For example, an operator might have manually closed the breaker at the wrong time but was not protected against this event because of a cultural tolerance for out of service equipment (such as synch check relays, the computer, etc.). I do not know what happened but I am confident many barriers that had to be bypassed for a catastrophic such as the Splendor event to occur. It was certainly not an act of god and not a single equipment or human failure. These events never are anymore than it was the fault of the iceberg that the Titanic struck for being in the wrong place at the wrong time.

The mechanical forces created by closing the breaker out of synch are huge when the incoming machine is small, relative to the total generation on the bus. In essence, enormous magnetic forces will instantly be imposed on the rotor that will try to instantly accelerate it to a synchronized position. I did not see it but I know of a 150 MVA generator that ended up significantly out of synch – the forces tore it apart, including shearing the common shaft.

Have a look on Coast Guard notice for fire related problems on ”Carnival Splendor”.
very alarming but not surprising; if electrical equipment was designed the same, all these ship are disaster waiting to happen.

Over two months have passed and the silence about what happened aboard the Splendor is deafening. I think any reasonable person has to come to the conclusion Carnival has a lot to hide.

Please see the U.S. Coast Guard safety alert in the following link.

http://www.marinelink.com/news/inoperative-failures336547.aspx

This is really disturbing to me because multiple failures were present in a recently inspected vital safety system on a relatively new vessel. This is not a case of a single unexpected failure in the CO2 system. It is a case where many issues concurrently existed in the CO2 system and each of these issues had a consequential adverse impact on the operability of the system. The root cause analysis (if one is even done) of the failure of the CO2 system would probably reveal some very significant systemic (administrative & cultural) issues.

It is not reasonable to assume that the CO2 system is unique and is the only system in this condition. The same management/work culture that allowed the CO2 system to remain inoperable affects everything aboard the vessel. What statement does this make about the overall safety of all cruise ships? What statement does this make about the effectiveness of the regulating bodies (e.g. the Coast Guard) to effectively regulate and protect the safety of the public? What can one conclude about every cruise ship now at sea?

I certainly never have been and am not now a “white knuckle flyer”. However, the double failure of the cruise line and the regulators to assure a vital safety system is operable is certainly going to force me to re-evaluate the safety of taking a cruise.

Please understand my perspective of how to run a safe (and reliable) engineering plant is from almost 40 years in the nuclear power industry. However, I see no material difference in the nuclear industry’s very real accountability to assure the health and safety of the public and the cruise industry’s accountability to assure the health and safety of almost 5000 passengers and crew. Even without an accident, if we found a “safety related” system in the same condition as that reported by the Coast Guard for the CO2 system, we would make international news, almost instantly have federal inspection teams on site, be subject to huge fines, and be forced into audits that could ultimately cost a few hundred million dollars to successfully deal with.

Why is the press and public now so silent about a problem that potentially effects the safety of thousands of people traveling on cruise ships right now?

Carnival Splendor is underway from Long Beach to San Francisco 1/22/11 approx. 1:45PM PST. Information taken from the cruise director’s blog on Facebook and
http://marinetraffic.com

One poster claimed a generator and two alternators need to be replaced. I cannot confirm that.

The Cruise Director is scheduled to arrive at pier 35 in San Francisco on February 12 to accompany the ship out on sea trials on February 14 and then on to Long Beach, arriving on February 19 apparently ready to take on passengers for the scheduled February 20 sailing.
THe press and or public simply are not following this saga, perhaps because it appeared that it was a “harmless” incident, with no potemtial for harming passengers or crew. If an equivalent accident occured on an airplane, all hell would have broken loose. Where are the US investgative bodies that should be monitoring and reporting on this incident?

Clear

Antonov delivers Engine for Carnival Splendor

“The engine was flown out from the manufacturer Wartsila via Venice, Italy and greeted by Blue Water (www.bws.dk) representive Roy Applegate who is over seeing it’s Installation.”

http://www.flickr.com/photos/wjcendak/5379933209/in/set-72157625899503788/

Rick, thanks for the links to the photos.

Also on the flikr photos, you can select the original large size. In this case they are very large sharp pics you can scan thru and see lot of detail in the engine room. Papers on the wall, wiring, ropes hanging. All of it looks relatively unscathed.

“Carnival Splendor Resumes Service – Cause of Fire Remains Unexplained”

http://articles.maritimepropulsion.com/article/Carnival-Splendor-Resumes-Service-e28093-Cause-of-Fire-Remains-Unexplained37006.aspx

“The fire in the aft engine-room spread extensively by all accounts, suggesting that a thin spread of oil from #5 diesel, thrown out with explosive force over a wide area, had to be dealt with; an outcome typical of an un-contained crankcase explosion. If this much is true, statistics from a leading classification society add credibility.

“Carnival Splendor has a diesel-electric propulsion system with medium-speed diesel engines (in separate engine rooms) as prime movers. According to an eleven-year analysis of crankcase explosions in its classed fleet from 1990, Lloyds Register recorded that 85% of a total of 143 such incidents in that period occurred in that type of diesel engine (the small remainder in two-stroke engines). Lloyds Register found the causes were due to bearings (39 per cent), pistons (47 per cent) and others (14 per cent).

“It may be that the engine-room fire aboard Carnival Splendor last November was caused by a crankcase explosion through over-heating or mechanical failure of one of the internals mentioned above within #5 diesel engine, but we shall have to wait and see. Place your bets!”

I was out on the ocean at the time, within 50 miles of where this ship was disabled. That same morning an unidentified missile was seen emerging from the sea, presumably from a submarine, about 20 miles southwest of San Diego. The missile veered slightly south and west as it rose. Many others on my vessel witnessed this. Later that same day the reports started about the disabled cruise ship. My initial thought is still the best theory (in my mind): a Chinese submarine tested an EMP weapon on this unsuspecting cruise ship, and it was hugely successful. There has been a cover-up so as not to panic the maritime community and general public.