| 1 | <html><head><META http-equiv="Content-Type" content="text/html; charset=iso-8859-1"><title>Apache Tribes - The Tomcat Cluster Communication Module - Apache Tribes - Introduction</title><meta value="Filip Hanik" name="author"><meta value="fhanik@apache.org" name="email"></head><body vlink="#525D76" alink="#525D76" link="#525D76" text="#000000" bgcolor="#ffffff"><table cellspacing="0" width="100%" border="0"><!--PAGE HEADER--><tr><td><!--PROJECT LOGO--><a href="http://tomcat.apache.org/"><img border="0" alt="Apache Tomcat" align="right" src="../images/tomcat.gif"></a></td><td><font face="arial,helvetica,sanserif"><h1>Apache Tomcat 6.0</h1></font></td><td><!--APACHE LOGO--><a href="http://www.apache.org/"><img border="0" alt="Apache Logo" align="right" src="../images/asf-logo.gif"></a></td></tr></table><table cellspacing="4" width="100%" border="0"><!--HEADER SEPARATOR--><tr><td colspan="2"><hr size="1" noshade></td></tr><tr><!--LEFT SIDE NAVIGATION--><td nowrap="true" valign="top" width="20%"><p><strong>Links</strong></p><ul><li><a href="../index.html">Docs Home</a></li><li><a href="http://tomcat.apache.org/faq">FAQ</a></li></ul><p><strong>User Guide</strong></p><ul><li><a href="introduction.html">1) Introduction</a></li><li><a href="setup.html">2) Setup</a></li><li><a href="faq.html">3) FAQ</a></li></ul><p><strong>Reference</strong></p><ul><li><a href="RELEASE-NOTES.txt">Release Notes</a></li><li><a href="/api/index.html">JavaDoc</a></li></ul><p><strong>Apache Tribes Development</strong></p><ul><li><a href="membership.html">Membership</a></li><li><a href="transport.html">Transport</a></li><li><a href="interceptors.html">Interceptors</a></li><li><a href="status.html">Status</a></li><li><a href="developers.html">Developers</a></li></ul></td><!--RIGHT SIDE MAIN BODY--><td align="left" valign="top" width="80%"><table cellspacing="4" width="100%" border="0"><tr><td valign="top" align="left"><h1>Apache Tribes - The Tomcat Cluster Communication Module</h1><h2>Apache Tribes - Introduction</h2></td><td nowrap="true" valign="top" align="right"><small><a href="printer/tribes.html"><img alt="Printer Friendly Version" border="0" src="../images/printer.gif"><br>print-friendly<br>version |
|---|
| 2 | </a></small></td></tr></table><table cellpadding="2" cellspacing="0" border="0"><tr><td bgcolor="#525D76"><font face="arial,helvetica.sanserif" color="#ffffff"><a name="Quick Start"><strong>Quick Start</strong></a></font></td></tr><tr><td><blockquote> |
|---|
| 3 | |
|---|
| 4 | <p>Apache Tribes is a group or peer-to-peer communcation framework that enables you to easily connect |
|---|
| 5 | your remote objects to communicate with each other. |
|---|
| 6 | </p> |
|---|
| 7 | <ul> |
|---|
| 8 | <li>Import: <code>org.apache.catalina.tribes.Channel</code></li> |
|---|
| 9 | <li>Import: <code>org.apache.catalina.tribes.Member</code></li> |
|---|
| 10 | <li>Import: <code>org.apache.catalina.tribes.MembershipListener</code></li> |
|---|
| 11 | <li>Import: <code>org.apache.catalina.tribes.ChannelListener</code></li> |
|---|
| 12 | <li>Import: <code>org.apache.catalina.tribes.group.GroupChannel</code></li> |
|---|
| 13 | <li>Create a class that implements: <code>org.apache.catalina.tribes.ChannelListener</code></li> |
|---|
| 14 | <li>Create a class that implements: <code>org.apache.catalina.tribes.MembershipListener</code></li> |
|---|
| 15 | <li>Simple class to demonstrate how to send a message: |
|---|
| 16 | <div align="left"><table border="0" cellpadding="0" cellspacing="4"><tr><td height="1" width="1" bgcolor="#023264"><img border="0" hspace="0" vspace="0" height="1" width="1" src="../images/void.gif"></td><td height="1" bgcolor="#023264"><img border="0" hspace="0" vspace="0" height="1" width="1" src="../images/void.gif"></td><td height="1" width="1" bgcolor="#023264"><img border="0" hspace="0" vspace="0" height="1" width="1" src="../images/void.gif"></td></tr><tr><td width="1" bgcolor="#023264"><img border="0" hspace="0" vspace="0" height="1" width="1" src="../images/void.gif"></td><td height="1" bgcolor="#ffffff"><pre> |
|---|
| 17 | //create a channel |
|---|
| 18 | Channel myChannel = new GroupChannel(); |
|---|
| 19 | |
|---|
| 20 | //create my listeners |
|---|
| 21 | ChannelListener msgListener = new MyMessageListener(); |
|---|
| 22 | MembershipListener mbrListener = new MyMemberListener(); |
|---|
| 23 | |
|---|
| 24 | //attach the listeners to the channel |
|---|
| 25 | myChannel.addMembershipListener(mbrListener); |
|---|
| 26 | myChannel.addChannelListener(msgListener); |
|---|
| 27 | |
|---|
| 28 | //start the channel |
|---|
| 29 | myChannel.start(Channel.DEFAULT); |
|---|
| 30 | |
|---|
| 31 | //create a message to be sent, message must implement java.io.Serializable |
|---|
| 32 | //for performance reasons you probably want them to implement java.io.Externalizable |
|---|
| 33 | Serializable myMsg = new MyMessage(); |
|---|
| 34 | |
|---|
| 35 | //retrieve my current members |
|---|
| 36 | Member[] group = myChannel.getMembers(); |
|---|
| 37 | |
|---|
| 38 | //send the message |
|---|
| 39 | channel.send(group,myMsg,Channel.SEND_OPTIONS_DEFAULT); |
|---|
| 40 | </pre></td><td width="1" bgcolor="#023264"><img border="0" hspace="0" vspace="0" height="1" width="1" src="../images/void.gif"></td></tr><tr><td height="1" width="1" bgcolor="#023264"><img border="0" hspace="0" vspace="0" height="1" width="1" src="../images/void.gif"></td><td height="1" bgcolor="#023264"><img border="0" hspace="0" vspace="0" height="1" width="1" src="../images/void.gif"></td><td height="1" width="1" bgcolor="#023264"><img border="0" hspace="0" vspace="0" height="1" width="1" src="../images/void.gif"></td></tr></table></div> |
|---|
| 41 | </li> |
|---|
| 42 | </ul> |
|---|
| 43 | <p> |
|---|
| 44 | Simple yeah? There is a lot more to Tribes than we have shown, hopefully the docs will be able |
|---|
| 45 | to explain more to you. Remember, that we are always interested in suggestions, improvements, bug fixes |
|---|
| 46 | and anything that you think would help this project. |
|---|
| 47 | </p> |
|---|
| 48 | <p> |
|---|
| 49 | Note: Tribes is currently built for JDK1.5, you can run on JDK1.4 by a small modifications to locks used from the <code>java.util.concurrent</code> package. |
|---|
| 50 | </p> |
|---|
| 51 | </blockquote></td></tr></table><table cellpadding="2" cellspacing="0" border="0"><tr><td bgcolor="#525D76"><font face="arial,helvetica.sanserif" color="#ffffff"><a name="What is Tribes"><strong>What is Tribes</strong></a></font></td></tr><tr><td><blockquote> |
|---|
| 52 | <p> |
|---|
| 53 | Tribes is a messaging framework with group communication abilities. Tribes allows you to send and receive |
|---|
| 54 | messages over a network, it also allows for dynamic discovery of other nodes in the network.<br> |
|---|
| 55 | And that is the short story, it really is as simple as that. What makes Tribes useful and unique will be |
|---|
| 56 | described in the section below.<br> |
|---|
| 57 | </p> |
|---|
| 58 | <p> |
|---|
| 59 | The Tribes module was started early 2006 and a small part of the code base comes from the clustering module |
|---|
| 60 | that has been existing since 2003 or 2004. |
|---|
| 61 | The current cluster implementation has several short comings and many work arounds were created due |
|---|
| 62 | to the complexity in group communication. Long story short, what should have been two modules a long time |
|---|
| 63 | ago, will be now. Tribes takes out the complexity of messaging from the replication module and becomes |
|---|
| 64 | a fully independent and highly flexible group communication module.<br> |
|---|
| 65 | </p> |
|---|
| 66 | <p> |
|---|
| 67 | In Tomcat the old <code>modules/cluster</code> has now become <code>modules/groupcom</code>(Tribes) and |
|---|
| 68 | <code>modules/ha</code> (replication). This will allow development to proceed and let the developers |
|---|
| 69 | focus on the issues they are actually working on rather than getting boggled down in details of a module |
|---|
| 70 | they are not interested in. The understanding is that both communication and replication are complex enough, |
|---|
| 71 | and when trying to develop them in the same module, well you know, it becomes a cluster :)<br> |
|---|
| 72 | </p> |
|---|
| 73 | <p> |
|---|
| 74 | Tribes allows for guaranteed messaging, and can be customized in many ways. Why is this important?<br> |
|---|
| 75 | Well, you as a developer want to know that the messages you are sending are reaching their destination. |
|---|
| 76 | More than that, if a message doesn't reach its destination, the application on top of Tribes will be notified |
|---|
| 77 | that the message was never sent, and what node it failed. |
|---|
| 78 | </p> |
|---|
| 79 | |
|---|
| 80 | </blockquote></td></tr></table><table cellpadding="2" cellspacing="0" border="0"><tr><td bgcolor="#525D76"><font face="arial,helvetica.sanserif" color="#ffffff"><a name="Why another messaging framework"><strong>Why another messaging framework</strong></a></font></td></tr><tr><td><blockquote> |
|---|
| 81 | <p> |
|---|
| 82 | I am a big fan of reusing code and would never dream of developing something if someone else has already |
|---|
| 83 | done it and it was available to me and the community I try to serve.<br> |
|---|
| 84 | When I did my research to improve the clustering module I was constantly faced with a few obstacles:<br> |
|---|
| 85 | 1. The framework wasn't flexible enough<br> |
|---|
| 86 | 2. The framework was licensed in a way that neither I nor the community could use it<br> |
|---|
| 87 | 3. Several features that I needed were missing<br> |
|---|
| 88 | 4. Messaging was guaranteed, but no feedback was reported to me<br> |
|---|
| 89 | 5. The semantics of my message delivery had to be configured before runtime<br> |
|---|
| 90 | And the list continues... |
|---|
| 91 | </p> |
|---|
| 92 | <p> |
|---|
| 93 | So I came up with Tribes, to address these issues and other issues that came along. |
|---|
| 94 | When designing Tribes I wanted to make sure I didn't lose any of the flexibility and |
|---|
| 95 | delivery semantics that the existing frameworks already delivered. The goal was to create a framework |
|---|
| 96 | that could do everything that the others already did, but to provide more flexibility for the application |
|---|
| 97 | developer. In the next section will give you the high level overview of what features tribes offers or will offer. |
|---|
| 98 | </p> |
|---|
| 99 | </blockquote></td></tr></table><table cellpadding="2" cellspacing="0" border="0"><tr><td bgcolor="#525D76"><font face="arial,helvetica.sanserif" color="#ffffff"><a name="Feature Overview"><strong>Feature Overview</strong></a></font></td></tr><tr><td><blockquote> |
|---|
| 100 | <p> |
|---|
| 101 | To give you an idea of the feature set I will list it out here. |
|---|
| 102 | Some of the features are not yet completed, if that is the case they are marked accordingly. |
|---|
| 103 | </p> |
|---|
| 104 | <p> |
|---|
| 105 | <b>Pluggable modules</b><br> |
|---|
| 106 | Tribes is built using interfaces. Any of the modules or components that are part of Tribes can be swapped out |
|---|
| 107 | to customize your own Tribes implementation. |
|---|
| 108 | </p> |
|---|
| 109 | <p> |
|---|
| 110 | <b>Guaranteed Messaging</b><br> |
|---|
| 111 | In the default implementation of Tribes uses TCP for messaging. TCP already has guaranteed message delivery |
|---|
| 112 | and flow control built in. I believe that the performance of Java TCP, will outperform an implementation of |
|---|
| 113 | Java/UDP/flow-control/message guarantee since the logic happens further down the stack.<br> |
|---|
| 114 | Tribes supports both non-blocking and blocking IO operations. The recommended setting is to use non blocking |
|---|
| 115 | as it promotes better parallelism when sending and receiving messages. The blocking implementation is available |
|---|
| 116 | for those platforms where NIO is still a trouble child. |
|---|
| 117 | </p> |
|---|
| 118 | <p> |
|---|
| 119 | <b>Different Guarantee Levels</b><br> |
|---|
| 120 | There are three different levels of delivery guarantee when a message is sent.<br> |
|---|
| 121 | <ol> |
|---|
| 122 | <li>IO Based send guarantee. - fastest, least reliable<br> |
|---|
| 123 | This means that Tribes considers the message transfer to be successful |
|---|
| 124 | if the message was sent to the socket send buffer and accepted.<br> |
|---|
| 125 | On blocking IO, this would be <code>socket.getOutputStream().write(msg)</code><br> |
|---|
| 126 | On non blocking IO, this would be <code>socketChannel.write()</code>, and the buffer byte buffer gets emptied |
|---|
| 127 | followed by a <code>socketChannel.read()</code> to ensure the channel still open. |
|---|
| 128 | The <code>read()</code> has been added since <code>write()</code> will succeed if the connection has been "closed" |
|---|
| 129 | when using NIO. |
|---|
| 130 | </li> |
|---|
| 131 | <li>ACK based. - recommended, guaranteed delivery<br> |
|---|
| 132 | When the message has been received on a remote node, an ACK is sent back to the sender, |
|---|
| 133 | indicating that the message was received successfully. |
|---|
| 134 | </li> |
|---|
| 135 | <li>SYNC_ACK based. - guaranteed delivery, guaranteed processed, slowest<br> |
|---|
| 136 | When the message has been received on a remote node, the node will process |
|---|
| 137 | the message and if the message was processed successfully, an ACK is sent back to the sender |
|---|
| 138 | indicating that the message was received and processed successfully. |
|---|
| 139 | If the message was received, but processing it failed, an ACK_FAIL will be sent back |
|---|
| 140 | to the sender. This is a unique feature that adds an incredible amount value to the application |
|---|
| 141 | developer. Most frameworks here will tell you that the message was delivered, and the application |
|---|
| 142 | developer has to build in logic on whether the message was actually processed properly by the application |
|---|
| 143 | on the remote node. If configured, Tribes will throw an exception when it receives an ACK_FAIL |
|---|
| 144 | and associate that exception with the member that didn't process the message. |
|---|
| 145 | </li> |
|---|
| 146 | </ol> |
|---|
| 147 | You can of course write even more sophisticated guarantee levels, and some of them will be mentioned later on |
|---|
| 148 | in the documentation. One mentionable level would be a 2-Phase-Commit, where the remote applications don't receive |
|---|
| 149 | the message until all nodes have received the message. Sort of like a all-or-nothing protocol. |
|---|
| 150 | </p> |
|---|
| 151 | <p> |
|---|
| 152 | <b>Per Message Delivery Attributes</b><br> |
|---|
| 153 | Perhaps the feature that makes Tribes stand out from the crowd of group communication frameworks. |
|---|
| 154 | Tribes enables you to send to decide what delivery semantics a message transfer should have on a per |
|---|
| 155 | message basis. Meaning, that your messages are not delivered based on some static configuration |
|---|
| 156 | that remains fixed after the message framework has been started.<br> |
|---|
| 157 | To give you an example of how powerful this feature is, I'll try to illustrate it with a simple example. |
|---|
| 158 | Imagine you need to send 10 different messsages, you could send the the following way: |
|---|
| 159 | <div align="left"><table border="0" cellpadding="0" cellspacing="4"><tr><td height="1" width="1" bgcolor="#023264"><img border="0" hspace="0" vspace="0" height="1" width="1" src="../images/void.gif"></td><td height="1" bgcolor="#023264"><img border="0" hspace="0" vspace="0" height="1" width="1" src="../images/void.gif"></td><td height="1" width="1" bgcolor="#023264"><img border="0" hspace="0" vspace="0" height="1" width="1" src="../images/void.gif"></td></tr><tr><td width="1" bgcolor="#023264"><img border="0" hspace="0" vspace="0" height="1" width="1" src="../images/void.gif"></td><td height="1" bgcolor="#ffffff"><pre> |
|---|
| 160 | Message_1 - asynchronous and fast, no guarantee required, fire and forget |
|---|
| 161 | Message_2 - all-or-nothing, either all receivers get it, or none. |
|---|
| 162 | Message_3 - encrypted and SYNC_ACK based |
|---|
| 163 | Message_4 - asynchronous, SYNC_ACK and call back when the message is processed on the remote nodes |
|---|
| 164 | Message_5 - totally ordered, this message should be received in the same order on all nodes that have been |
|---|
| 165 | send totally ordered |
|---|
| 166 | Message_6 - asynchronous and totally ordered |
|---|
| 167 | Message_7 - RPC message, send a message, wait for all remote nodes to reply before returning |
|---|
| 168 | Message_8 - RPC message, wait for the first reply |
|---|
| 169 | Message_9 - RPC message, asynchronous, don't wait for a reply, collect them via a callback |
|---|
| 170 | Message_10- sent to a member that is not part of this group |
|---|
| 171 | </pre></td><td width="1" bgcolor="#023264"><img border="0" hspace="0" vspace="0" height="1" width="1" src="../images/void.gif"></td></tr><tr><td height="1" width="1" bgcolor="#023264"><img border="0" hspace="0" vspace="0" height="1" width="1" src="../images/void.gif"></td><td height="1" bgcolor="#023264"><img border="0" hspace="0" vspace="0" height="1" width="1" src="../images/void.gif"></td><td height="1" width="1" bgcolor="#023264"><img border="0" hspace="0" vspace="0" height="1" width="1" src="../images/void.gif"></td></tr></table></div> |
|---|
| 172 | As you can imagine by now, these are just examples. The number of different semantics you can apply on a |
|---|
| 173 | per-message-basis is almost limitless. Tribes allows you to set up to 28 different on a message |
|---|
| 174 | and then configure Tribes to what flag results in what action on the message.<br> |
|---|
| 175 | Imagine a shared transactional cache, probably >90% are reads, and the dirty reads should be completely |
|---|
| 176 | unordered and delivered as fast as possible. But transactional writes on the other hand, have to |
|---|
| 177 | be ordered so that no cache gets corrupted. With tribes you would send the write messages totally ordered, |
|---|
| 178 | while the read messages you simple fire to achieve highest throughput.<br> |
|---|
| 179 | There are probably better examples on how this powerful feature can be used, so use your imagination and |
|---|
| 180 | your experience to think of how this could benefit you in your application. |
|---|
| 181 | </p> |
|---|
| 182 | <p> |
|---|
| 183 | <b>Interceptor based message processing</b><br> |
|---|
| 184 | Tribes uses a customizable interceptor stack to process messages that are sent and received.<br> |
|---|
| 185 | <i>So what, all frameworks have this!</i><br> |
|---|
| 186 | Yes, but in Tribes interceptors can react to a message based on the per-message-attributes |
|---|
| 187 | that are sent runtime. Meaning, that if you add a encryption interceptor that encrypts message |
|---|
| 188 | you can decide if this interceptor will encrypt all messages, or only certain messages that are decided |
|---|
| 189 | by the applications running on top of Tribes.<br> |
|---|
| 190 | This is how Tribes is able to send some messages totally ordered and others fire and forget style |
|---|
| 191 | like the example above.<br> |
|---|
| 192 | The number of interceptors that are available will keep growing, and we would appreciate any contributions |
|---|
| 193 | that you might have. |
|---|
| 194 | </p> |
|---|
| 195 | <p> |
|---|
| 196 | <b>Threadless Interceptor stack</b> |
|---|
| 197 | The interceptor don't require any separate threads to perform their message manipulation.<br> |
|---|
| 198 | Messages that are sent will piggy back on the thread that is sending them all the way through transmission. |
|---|
| 199 | The exception is the <code>MessageDispatchInterceptor</code> that will queue up the message |
|---|
| 200 | and send it on a separate thread for asynchronous message delivery. |
|---|
| 201 | Messages received are controlled by a thread pool in the <code>receiver</code> component.<br> |
|---|
| 202 | The channel object can send a <code>heartbeat()</code> through the interceptor stack to allow |
|---|
| 203 | for timeouts, cleanup and other events.<br> |
|---|
| 204 | The <code>MessageDispatchInterceptor</code> is the only interceptor that is configured by default. |
|---|
| 205 | </p> |
|---|
| 206 | <p> |
|---|
| 207 | <b>Parallel Delivery</b><br> |
|---|
| 208 | Tribes support parallel delivery of messages. Meaning that node_A could send three messages to node_B in |
|---|
| 209 | parallel. This feature becomes useful when sending messages with different delivery semantics. |
|---|
| 210 | Otherwise if Message_1 was sent totally ordered, Message_2 would have to wait for that message to complete.<br> |
|---|
| 211 | Through NIO, Tribes is also able to send a message to several receivers at the same time on the same thread. |
|---|
| 212 | </p> |
|---|
| 213 | <p> |
|---|
| 214 | <b>Silent Member Messaging</b><br> |
|---|
| 215 | With Tribes you are able to send messages to members that are not in your group. |
|---|
| 216 | So by default, you can already send messages over a wide area network, even though the dynamic discover |
|---|
| 217 | module today is limited to local area networks by using multicast for dynamic node discovery. |
|---|
| 218 | Of course, the membership component will be expanded to support WAN memberships in the future. |
|---|
| 219 | But this is very useful, when you want to hide members from the rest of the group and only communicate with them |
|---|
| 220 | </p> |
|---|
| 221 | </blockquote></td></tr></table><table cellpadding="2" cellspacing="0" border="0"><tr><td bgcolor="#525D76"><font face="arial,helvetica.sanserif" color="#ffffff"><a name="Where can I get Tribes"><strong>Where can I get Tribes</strong></a></font></td></tr><tr><td><blockquote> |
|---|
| 222 | <p> |
|---|
| 223 | |
|---|
| 224 | </p> |
|---|
| 225 | |
|---|
| 226 | |
|---|
| 227 | </blockquote></td></tr></table></td></tr><!--FOOTER SEPARATOR--><tr><td colspan="2"><hr size="1" noshade></td></tr><!--PAGE FOOTER--><tr><td colspan="2"><div align="center"><font size="-1" color="#525D76"><em> |
|---|
| 228 | Copyright © 1999-2008, Apache Software Foundation |
|---|
| 229 | </em></font></div></td></tr></table></body></html> |
|---|
