TremSnaps 1.0

[Syntac]

TremSnaps is a web app that captures the state of the entire Tremulous network. The state of each server, and each player, is written to a snapshot that can then be viewed by visitors. One snapshot is about 150 KB.

Installation
Just unzip it into whatever directory you like. Make sure PHP has full read/write access to the entire directory, or weird things can happen.

Automatic Snapshots
TremSnaps doesn't take snapshots on its own. You'll have to set up a cron job or something equivalent.

Code:

0 * * * * tremsnaps/snapshot.php

This would generate a snapshot every hour.

Configuring
The configuration file is common/config.php.

Downloads
Browse

Screenshots
Viewing a snapshot:

Viewing DretchStorm:

[Bissig]

One Megabyte? Can't you compress it or use another method for data storage?

Looks promising. I'll have a look at it later today.

[David]

What the hell are you storing?
qstat -R -P only comes out to 141k, 24k gzipped.

[Syntac]

Crap, I was way off. It's about 150 KB. Turns out Mac OS X uses screwy estimation and you have to look at the number of bytes in order to find the real size.

[Ellohir]

What does "server with stupid names" mean?

[Syntac]

Heh, I was hoping someone would ask that. You know how some server owners put black "A"s in front of the title? Well, TremSnaps detects that.

It also removes boxes and deals with nonstandard color escapes.

[Colynn']

Hello,
Excuse-me of my newbieness, but does it works on Linux/Windows ?
PS : And for the necro too.

[Bissig]

Hello,
Excuse-me of my newbieness, but does it works on Linux/Windows ?
PS : And for the necro too.

It's a php script - it works on almost any OS (well, at least on those where Tremulous runs on)

[Syntac]

It works on all platforms that support the Apache (2?) web server and PHP 5.

[Colynn']

Works!

[Syntac]

Did you set up a cron job?

[EDIT] Rawr, typo.

[Colynn']

Nop, sorry. Actually, I'm on windows (I have Ubuntu too, but I need to fix some things before) and I don't know how to set up a

Code:

cron

on it.
And it's not my own server, it's the one of Symphony/Haptism, a french (??), who hosts me.

[Syntac]

Ohhh... Well, I'm just glad someone was able to get TremSnaps working.

[Archangel]

Code:

[intellekt@oceanlab.ezbake.org ~/public_html/tremsnaps/] % unzip TremSnaps-R001.zip
Archive:  TremSnaps-R001.zip
   creating: tremsnaps/
  inflating: tremsnaps/.DS_Store     
   creating: __MACOSX/
   creating: __MACOSX/tremsnaps/
  inflating: __MACOSX/tremsnaps/._.DS_Store 
   creating: tremsnaps/common/
  inflating: tremsnaps/common/.DS_Store 
   creating: __MACOSX/tremsnaps/common/
  inflating: __MACOSX/tremsnaps/common/._.DS_Store 
  inflating: tremsnaps/common/config.php 
  inflating: __MACOSX/tremsnaps/common/._config.php 
   creating: tremsnaps/common/css/
  inflating: tremsnaps/common/css/default.css 
   creating: __MACOSX/tremsnaps/common/css/
  inflating: __MACOSX/tremsnaps/common/css/._default.css 
   creating: tremsnaps/common/img/
  inflating: tremsnaps/common/img/favicon.png 
  inflating: tremsnaps/common/tremsnaps.php 
  inflating: __MACOSX/tremsnaps/common/._tremsnaps.php 
  inflating: tremsnaps/common/zendb.php 
  inflating: __MACOSX/tremsnaps/common/._zendb.php 
  inflating: tremsnaps/index.php     
  inflating: __MACOSX/tremsnaps/._index.php 
  inflating: tremsnaps/snapshot.php 
  inflating: __MACOSX/tremsnaps/._snapshot.php 
   creating: tremsnaps/snapshots/
  inflating: tremsnaps/snapshots/.DS_Store 
   creating: __MACOSX/tremsnaps/snapshots/
  inflating: __MACOSX/tremsnaps/snapshots/._.DS_Store 

[oceanlab.ezbake.org - 06:13:57 #22]
[intellekt@oceanlab.ezbake.org ~/public_html/tremsnaps/] % ls
total 22
drwxr-xr-x   4 intellekt  operator    512 Nov  1 06:13 .
drwxr-xr-x  10 intellekt  operator   1024 Nov  1 06:13 ..
-rw-r--r--   1 intellekt  operator  12794 Oct 18 09:03 TremSnaps-R001.zip
drwxrwxr-x   3 intellekt  operator    512 Oct 18 00:03 __MACOSX
drwxrwxrwx   4 intellekt  operator    512 Oct 18 00:00 tremsnaps

Wow, no need for the __MACOSX folder. Please fix permissions out of the box too!

http://oceanlab.ezbake.org/~intellekt/tremsnaps/

[Colynn']

I'm on it! o/

EDIT : Btw, you should remove the stupid/not correct server's name system. Because some servers who have a correct name are not listed...

[Syntac]

Aargh, there's more wrong with Mac OS X's zip program than I thought! I'll repackage it as a gzip less defective archive.

Btw, you should remove the stupid/not correct server's name system. Because some servers who have a correct name are not listed...

What do you mean?

[Amanieu]

Why is the Tremfusion Test Server not on the list

[Syntac]

Not sure. Could be due to packet loss.

[Archangel]

Yeah. there are a lot of servers that are fairly good that use boxes. I'd say just strip out the boxes, but still print the servers.

[Syntac]

It does. Boxes are stripped out when the list is retrieved.

[Archangel]

I see none of my servers, and they're all local to the machine.

[Syntac]

TremSnaps retrieves the list from the master server.

[Archangel]

I'm well aware. What I am saying is, none of my servers are showing up, and they can't possibly be timing out.

[Sap]

hehe ... my name is on top of that DretchStorm snapshot. Good timing Syntac  .

[Syntac]

I'm well aware. What I am saying is, none of my servers are showing up, and they can't possibly be timing out.

There could be packet loss that corrupts the list entry (and causes it to be removed), or something that prevents TremSnaps from connecting. Then there's the possibility of your servers not being public.

[Archangel]

The fact that they're on a 100mbit line, with a 20ms ping to the master server, and they haven't gone down for over 70 days, leads me to believe there is a bug in your code.

[Syntac]

There is very little possibility of there being a bug. This is all my code does:
    • Fetches a list from the master server (stripping out corrupted entries).
    • For each server on that list, connects to it and retrieves information.
    • Writes the accumulated data to a collection of files.
If you tell me what your servers' names are, I'll be happy to troubleshoot this myself.

[EDIT] Double negative...

[Undeference]

try

Code:

--- common/tremsnaps.php.orig 2008-11-01 21:08:44.000000000 -0400
+++ common/tremsnaps.php 2008-11-01 21:09:39.000000000 -0400
@@ -205,7 +205,9 @@
  $out = "\xFF\xFF\xFF\xFFgetservers 69 empty full";
 
  @fwrite( $fp, $out );
- $data_string = @fread( $fp, 100000 );
+ $data_string = '';
+ while( ( $read = fread( $fp, 100000 ) ) !== FALSE )
+ $data_string .= $read;
  $meta = @stream_get_meta_data( $fp );
  @fclose( $fp );
 

or some such

By the way, all the files in your current zip are gz'ed. Might want to fix that.

[Syntac]

Dammit, I know I'm in trouble when someone makes a diff of my code...

All that aside, thanks Undeference. Will give it a try.

[Syntac]

I will shortly be remaking TremSnaps into a command-line tool. Projected features include:
    • Scope options
    • Flat-file dumps
    • Rcon console
    • Dump readback
    • And probably more...

[Amanieu]

Servers will respond to your requests with more than 1 udp packet, you need to make sure you read all packets, no just the first. (This also applies to the master)

[Syntac]

I'll keep that in mind when working on the next implementation.

[David]

    • Fetches a list from the master server (stripping out corrupted entries).

If you have corrupted entries, then you're reading it wrong.
Also do you wait for all the packets to arrive or just read the first?  There's no way to tell how many will come short of waiting for a second or two.  At the moment there's nearly always two packets sent, although that can change with the number of servers.

EDIT: Didn't see page two, and if a server sends more than one then there's no way to safely reassemble them without guessing :s

[Archangel]

ding ding ding We have a winner.

[Einstein]

I just saw this thread, and I like what I see. I've been trying to make a Perl and PHP implementation of something like DASPRiD's TremStats, but with a lot more functionality, and most of my features thus far have been geared towards admins rather than the general public. I was wondering if you wouldn't like to join up, and we'll see what a couple of great minds can't come up with. I'm a little handy with PHP, but more so with C/C++ and Perl, so I could use your ideas. If you guys have any comments, I'd love to hear them; an outdated version of my project can be found at http://tremtools.googlecode.com. (I say outdated because there is no public collaboration yet, so the Google Code repository is way behind my local SVN repository that I keep my projects in.) Let me know.

By the way, when I unpack your zip I get absolute garbage; each file is intact but filled with some weird compressed or encoded content, not plaintext. I also realize that TremSnaps is far from finished, and I'd like to see it perfected; once that happens, I'd love to make aggregate statistics logging from it, so that we can see for all the servers in the master list what percent of the time which maps are played, what the balance is for hundreds of servers instead of just individual servers for each mod/gamename, what servers everyone frequents, etc. I could do all of this in Perl, of course, but it would be a royal pain in the ass, and the interface would just plain suck.

[Syntac]

Just so you know, the TremSnaps code is sloppiness in digital form. I intend to do a full redesign. The rewrite I mentioned earlier was sort of fail, due to its crippling inability to list more than twelve servers. Once I solve that problem, awesomeness should result.

By the way, when I unpack your zip I get absolute garbage; each file is intact but filled with some weird compressed or encoded content, not plaintext.

That's due to my somewhat misinfomed use of gzip. Never, ever do this...

Code:

gzip -r valuable_stuff/

...unless you want to spend a very long time decompressing everything. All I can say is, thank the Lord for Time Machine. The download should be okay now.

Anyway. What say you to a script that logs everything in a MySQL database?

[Einstein]

Anyway. What say you to a script that logs everything in a MySQL database?

That's exactly what I do now; perhaps you can help me with the front-end, at least, but yeah, my suite uses Perl's DBI (DataBase Interface) module to write and read directly to/from any SQL DB.

Oh, and unless you fixed the download in the past five minutes, it's still screwy.

[Syntac]

Should be fixed now. Re-uploaded it.

I don't know how easy it'll be to combine PHP and Perl into a single suite. Also, I don't know how many web hosts out there offer mod_perl (assuming this is going to be a web app).

Take this to e-mail, perhaps?

[Einstein]

Much better, it's actually text now, thanks.

Mod Web hosts have to offer Perl for CGI scripting, and besides, if all works out well it'll only have to execute PHP code; the Perl stuff is the backend, which ideally should only have to be run on the game server itself.

Either form of communication is fine. If you want to avoid spamming up Tremulous.net, you can post to the budding Google Group dedicated to my project, or simply e-mail me at one of the addresses that I use for my IM accounts, shown on my profile here. I cannot find your own e-mail address without doing some extensive research; your own site's contact page returns a 404 and you don't post your address as public here on the forums, so I guess you'll have to make first contact, eh?

[Syntac]

The 404 error is because I'm developing an e-mail script (yet another way of hiding my address).

I guess I'll have to trust you.

[Undeference]

I just saw this thread, and I like what I see. I've been trying to make a Perl and PHP implementation of something like DASPRiD's TremStats, but with a lot more functionality, and most of my features thus far have been geared towards admins rather than the general public.

Maybe I don't understand properly, but isn't the purpose of tremsnaps to query servers and store their responses, not to handle local server data (e.g., tremstats parses logs from the local server)? I really don't see much similarity... (Are you trying to do something more like http://betaserv.tk/?)

I was wondering if you wouldn't like to join up, and we'll see what a couple of great minds can't come up with. I'm a little handy with PHP, but more so with C/C++ and Perl, so I could use your ideas.

Wasn't DASPRiD planning on rewriting the tremstats parser in C?

If you guys have any comments, I'd love to hear them; an outdated version of my project can be found at http://tremtools.googlecode.com. (I say outdated because there is no public collaboration yet, so the Google Code repository is way behind my local SVN repository that I keep my projects in.) Let me know.

Since you asked for comments, I'll make a few:
Your coding style seems non-existent.
Comments are for explaining what unobvious code is doing or what you are doing with the code, not for burying it.
Your code does many things that will not work properly or at all (though you may have fixed them locally).
Your code does a lot of things suboptimally. That is an understatement.

[Einstein]

I just saw this thread, and I like what I see. I've been trying to make a Perl and PHP implementation of something like DASPRiD's TremStats, but with a lot more functionality, and most of my features thus far have been geared towards admins rather than the general public.

Maybe I don't understand properly, but isn't the purpose of tremsnaps to query servers and store their responses, not to handle local server data (e.g., tremstats parses logs from the local server)? I really don't see much similarity... (Are you trying to do something more like http://betaserv.game-host.org/server?)

Most of my stuff right now parses the logs, yes, and much of my focus has been on statistics of the server; imagine if I could add statistics from the entire Tremulous community by using the master list's (limited) information? And yes, to some extent, the betaserv's pages do a few of the same things that mine do, although I am not sure of their implementation methods.

I was wondering if you wouldn't like to join up, and we'll see what a couple of great minds can't come up with. I'm a little handy with PHP, but more so with C/C++ and Perl, so I could use your ideas.

Wasn't DASPRiD planning on rewriting the tremstats parser in C?

I have no idea, actually. Last time I checked his project was entirely dormant, and his site was all but totally inactive.

If you guys have any comments, I'd love to hear them; an outdated version of my project can be found at http://tremtools.googlecode.com. (I say outdated because there is no public collaboration yet, so the Google Code repository is way behind my local SVN repository that I keep my projects in.) Let me know.

Since you asked for comments, I'll make a few:
Your coding style seems non-existent.
Comments are for explaining what unobvious code is doing or what you are doing with the code, not for burying it.
Your code does many things that will not work properly or at all (though you may have fixed them locally).
Your code does a lot of things suboptimally. That is an understatement.

Explain what you'd like to be done better in terms of my "coding style," whether you mean method, syntax, lack of whitespace, etc.

I know that my comments are annoying as hell, at least for that one file. My original version wasn't commented at all, but I think that was before I created the Google Code repository. It actually took me longer to comment that mini-program than it did to code it, I think, but I did so for an express purpose, because I originally wrote it to be tested on the X server and an interested party there was not a coder and so I commented it so as to be understandable by a non-programmer. Because at that point it was not intended to be truly public, I did not take that into consideration; I have yet to clean it up, as you can see, even on my local revisions. The only reason I can read it is because of gedit's syntax highlighting that makes it neither helpful or harmful, but rather all but invisible if I so choose. In my programming environment, it's easy enough to ignore. When there's a ton of people that actually want to start looking at the code, I'll start writing it for them instead of for a couple of friends from X.

I honestly cannot remember where I was at when I last committed the Google Code copy of my code, but I can tell you right now that in some spots it's still like that, and my comments say as much. I intentionally partially code some things but then tell them to do nothing or nothing of great importance, leaving them until I can finish other related parts of the project that depend on them. This is very intentional and is merely an example of setting up functionality that doesn't yet exist. It does exactly what I want it to, however; otherwise, I fix it immediately, be assured of that.

I'm not sure if I mentioned this anywhere other than the X server forums, but I should say that I learned Perl for the sole purpose of writing the original logfile parser. I had never touched Perl before in my life, and am still learning, of course. It is a very versatile but sometimes very obscure language that I find both gratifying and frustrating at the same time. It also happens to be quite fast; not nearly as fast as C, of course, but rather well-performing nonetheless. So if by "optimally" you refer to "optimization" then I suppose that's why. If you'd care to be more specific, I'm sure there's something you can teach me that I'd be very grateful for!

The 404 error is because I'm developing an e-mail script (yet another way of hiding my address).

There's a thousand and one ways to do this, whether it's CGI, JavaScript, or simple encoding like URL escaping using the HTML hex symbols for each character. It should take all of ten seconds to do, if you want me to show you...

[Syntac]

Personally, I think this could be better implemented with PHP. Development might eventually start, when I have the time.

[Einstein]

Personally, I think this could be better implemented with PHP. Development might eventually start, when I have the time.

Er... How's that going to work? PHP is strictly server-side, and so as long as it eventually spits out your e-mail as a string, a spambot will see it. On the contrary, very few spambots execute JavaScript or other client-sided implementations. Anyways...

[Syntac]

Er... How's that going to work? PHP is strictly server-side, and so as long as it eventually spits out your e-mail as a string, a spambot will see it. On the contrary, very few spambots execute JavaScript or other client-sided implementations. Anyways...

I get the feeling we're talking about two completely different things.

PHP is strictly server-side

Wrong. It's also a command-line scripting language.

I was talking about TremSnaps or TremTools or whatever you want to call it. You seem to be talking about something I mentioned in passing (that also happens to be completely unrelated).

-- While we're still on the topic of e-mail: You don't seem to be very well-informed about PHP. My system allows me to present users with a form. When they submit it, their message is sent to my address via PHP's mail() function. Nothing is revealed to anyone.

[kevlarman]

I'm a little handy with PHP, but more so with C/C++ and Perl, so I could use your ideas.

you know you wanna do it in visual basic 

[Einstein]

My apologies, I thought you were talking about your site, and I didn't realize you were trying for more of a comment box, which is also pretty easy to do. I am well aware of PHP's capabilities, as I have been coding in it for many years, but mostly as a web dev rather than for local applications like this. I considered doing TremTools in either PHP (esp. for its database handling) or C++ (for its speed and object-oriented capabilities) and decided Perl was a nice compromise. Honestly, a big factor in that decision was not practicality; at the time, I was excessively bored, and needed to learn something new. Perl as the medium for TremTools seemed to fit the bill, long before it was TremTools.

Kevlarman, it's nice to see you again, after I've been MIA for a year. Your humor never fails to amuse me.

[Syntac]

My apologies, I thought you were talking about your site, and I didn't realize you were trying for more of a comment box, which is also pretty easy to do.

Seriously man, what in the name of all things holy are you talking about?

I will attempt to explain what I said in very simple terms.
    • It's a script that contains a form.
    • There's a text box.
    • Someone types something in the box.
    • And submits the form.
    • I receive an e-mail with the submitted text.
    • Not a comment box.

This is a comment box.

-- All that nonsense aside, I'll eventually get around to e-mailing you with some stuff (wow Syntac, that was descriptive).

[Einstein]

In the name of all things unholy, I've always thought of a comment box as something like Benmachine's, except it also e-mails you every post. I'm confused as to why this is difficult, you and I are talking about the same thing now... sorry for the confusion; apparently it's making me sound like an imbecile here. Anyways, wouldn't something like Matt Wright's FormMail or its equivalent fulfill your needs nicely? It has the necessary validation and security checks built into the Perl script...

Edit: I just checked DASPRiD's site. While he's apparently back to work on the site and has a Web presence elsewhere again, it's equally evident that TremStats has not been touched in a year. Oh, well.

[Syntac]

AAAAAAAAAHHHHH OFFTOPIC!!!!!!!!!!!!!!!!!

Anyways, wouldn't something like Matt Wright's FormMail or its equivalent fulfill your needs nicely? It has the necessary validation and security checks built into the Perl script...

No. Mainly because it's not written in PHP. Also because I only use my own projects, since it's easier for me to troubleshoot them.

[professor]

hmm well i just read this right now, and i cant really figure what to do with it once i download it.. do i just put it in a folder or sumthin cuz i dont get it..

[Syntac]

It's rather simple. You put the

in the

and you

it all together.

*sigh*

First, get your computer's built-in Apache web server working (with the PHP module enabled).

[professor]

o god i hate to do this to you, but i have no idea how to do that so ummm how do u do that?

[Syntac]

Google is your friend.

The PHP website has instructions (you'll want to read the comments at the bottom).

[professor]

hmm that wasnt very helpful, so can u please just help

[Syntac]

No, I can't "just help". Stop being a lazy ass and use a search engine.

[professor]

i did look:http://www.google.com/search?client=safari&rls=en-us&q=apache+web+server&ie=UTF-8&oe=UTF-8

[Syntac]

Obviously, you aren't very good at searching. Buy a brain and stop wasting my time.

[Bissig]

i did look:http://www.google.com/search?client=safari&rls=en-us&q=apache+web+server&ie=UTF-8&oe=UTF-8

Search for "selfphp" and "how to run a webserver with apache"

[professor]

o thanks that was helpful i think that works..

[Rocinante]

Obviously, you aren't very good at searching. Buy a brain and stop wasting my time.

You're wasting more time by posting this drivel.  Stop wasting my time (which is accurate, since I should give some common courtesy to tell you why your ass is gonna get banned and therefore I did have to post this, while your post didn't do anything for anyone).

o thanks that was helpful i think that works..

Good that this works, but if it doesn't, installing/configuring/running a PHP & Apache system is beyond the scope of this forum.  You're on the right track, namely judicious use of Google to find the answers (or your local bookstore).

[Syntac]

Hmm, usually no one stops my random flametrolling... Thanks, by the way. I would love not to get banned.

Anyway. I'll shortly be creating something similar to this, assuming my boss ever stops calling me about server trouble in the middle of the night.