Introducing Slipstream

[stp]

Hi. For the last several months I've been working part-time (along with help from many, especially Owen) on a project called Slipstream. It's still a little rough, but there's enough of an overall framework and guide that I thought I'd post about it to start getting some discussion and feedback going.

What is Slipstream? Really, it's two things. First, it's an attempt to capture the abstract experiences that many of us have had over the last few years into a set of programming patterns and best practices for Darkstar. Second, it's an effort to apply these patterns to creating a general-purpose programming framework that "herds" developers in the right direction and helps them build better, more scalable applications with more ease.

I'd like to invite anyone interested to go to the project home and grab the current trunk. In there you'll find the code (which compiles but can't be used to run a full game yet), a fairly detailed guide, and full code examples of the snippets from the guide.

Note that the first major section of the guide acts as a general introduction to some useful patterns and design goals for programming to Project Darkstar, so you might want to just skim through this section if you're looking for more details about general use of the server. Again, this is very much a work in progress, so there's probably plenty that's missing or confusing. Comments and contributions are very welcome!


seth

[nazica]

Very interesting! I suggest to move the first section with best practices to a separate document because this section must read every PDS-developer.

[vincent c.]

Interesting, thank you.

[adonies]

Haven't checked out the code (yet), but read the guide - which is, simply put, fantastic.

Apart from its immense usefulness as a guide, it contains tons of knowledge that only A*  search in the forums could provide.

And it's so well written, that I felt like "I wish I had a game concept ready to start coding?".

Really, I haven't felt so enthusiastic about PDS since the first day I read the PDS tutorial...

Thank you, Seth.

[thecreatrix]

Wow! This could really help lower the barriers to entry. I can imagine game starter kits based on this in the future.

And please continue to develop it further. My next game will very likely use Slipstream as a starting point (since my current one is too far along to rewrite).

Thank you for sharing!

[nokama]

Much needed and totally appreciated piece of work. I can imagine that it will focus the community greatly to have a common framework, which can be extended with reusable custom code, maybe a whole engine could grow from this one day..

To be honest, I think a good public MMO engine is what is required to really push Darkstar forward, I know it's a bit fantasy land, but maybe a few years down the line the community could come up with some of the advanced features of the big boys, say HeroEngine or Big World. That would be proof to the naysayers that Darkstar can really cut it

The more I delve into game development, the less anamoured I am with the prospect of building and running a game, there's soo much responsibility that goes along with it, not to mention finding or creating good content. I think I'd rather research and develop Darkstar compatible modules, that would be fun, and Slipstream is the perfect centre point to tie in with.

Imagine a GUI development client, scripting, animation, terrain editing etc, maybe deployment system for new server code, a complete system would be amazing. Obviously there are so many ways to achieve this, but one standardized engine could be modded etc, it'd still cut down development time considerably.

[No]

This is a really good piece of work I wished I had before ! But it's never to late to read this, and I'm glad we've followed most of the practices described here.

However, I have questions about two of your advices in the document.

 - "Prefer types to ManagedReference" : here you say it's better to use type for function parameters than reference. I think I understand the code maintenance reason but I always choose it was best to pass managedReference in order to avoid potential useless (and costly) dereferencing. So may I summarize that passing managedreference is discouraged for code maintenance but better for performance or am I wrong ?
I also though about using types only if the parameter will always be used and managedreference if it may not but I choose to keep same practice everywhere to facilitate comprehension...

 - About channels : you say having a channel with every connected client is not a good practice. So my question is, what is the best way to send a message to every connected client ?
=> Channel(obviously not)
=> scalableList
=> any kind of method I don't know to retrieve every object of the same kind from the datastore ? (something like AppContext.getDataManager().getAll(MyClass.class)

One more, thanks for this document. Despite the fact it's a bit late to use this project to help our project developpement, I will follow it and surely take some good practices out of it.

[stp]

Hi to all on this thread. Glad this is proving to be useful!

- "Prefer types to ManagedReference" : here you say it's better to use type for function parameters than reference. I think I understand the code maintenance reason but I always choose it was best to pass managedReference in order to avoid potential useless (and costly) dereferencing. So may I summarize that passing managedreference is discouraged for code maintenance but better for performance or am I wrong ?
I also though about using types only if the parameter will always be used and managedreference if it may not but I choose to keep same practice everywhere to facilitate comprehension...

That's a good question. My experience is that generally when you're passing some value, it's because you already have dereferenced it (or you're just creating it in the current transaction). That's certainly what the Slipstream design tends to enforce. This is really what you're getting at in the second part of your question above. Yes, it's definitely costly to dereference an object only to pass it to another object, but in practice I don't come across many cases where neither the sender nor the receiver needs access this object.

When this does happen, it's because multiple objects have been fetched from the data store already, and now they need to coordinate on some other, "passive" object. Again, my experience is that these scenarios are pretty infrequent, so the overhead of fetching the object here is usually acceptable, but you'd have to decide how you want to design it. Generally I find that designing for minimal object access tends to keep you away from these kinds of use-cases, but like I said in the doc, "prefer" this design, don't do it blindly

- About channels : you say having a channel with every connected client is not a good practice. So my question is, what is the best way to send a message to every connected client ?

I'm not sure that Darkstar has an one obvious way to handle this. The way that I tend to manage player state is with name-bindings. These names are kept in their own namespace, so it's possible to iterate over all players. You could write a task that used the shouldCountinue() method to iterate over all players, looking for the currently connected players, and send each one your message.

I guess I'd like to turn this around and ask what you need this mechanism for. I haven't really come across many scenarios where you want to broadcast a message to every player in the world. Broadcasts to everyone in some section of a game, sure, but not all connected players. Can you give me some examples of what you're trying to support?

Like I said above, I'm glad this is proving useful! Please keep the questions coming..


seth

[thecreatrix]

I haven't really come across many scenarios where you want to broadcast a message to every player in the world. Broadcasts to everyone in some section of a game, sure, but not all connected players. Can you give me some examples of what you're trying to support?

If I can jump in here, one thing I can think of is to tell all clients that the server will be going down or to actually kick all clients off when it does happen. Or any other kind of game-wide announcement, like saying that a special item just went on sale in the item shop for the next 30 minutes only.

[Tim B]

That was my thinking to thecreatrix - any sort of systemwide message sent out to everyone connected.

However I've been thinking that that could be handled outside of PDS using a news ticker type system....clients could poll the news page every now and then and display the alert when it appears there.

That also means that people just visiting the website could get alerts/information without needing to have the game actively running at the time.

It's just a thought at the moment though!

[stp]

However I've been thinking that that could be handled outside of PDS using a news ticker type system....clients could poll the news page every now and then and display the alert when it appears there.

I'm with Tim B on this. I would tend to model any system-wide event as something outside of Darkstar, precisely so that it could be observed from other applications or from players who aren't currently logged in. It could be a pull or a push mechanism depending on how you wanted to implement it. A web service or the like would be my choice, but I can think of many protocols that would be ideal for this kind of notification. Just my personal preference..


seth

[No]

Mmmm. In my case I need to send message to every client to handle global time event (we have a day based gameplay and may expect some meteorogical event).

But I can think about many other needs for such broadcast as thecreatrix said, gm events, annoucement... And I don't think a poll system would be better in performance.

So I will use this great new shouldContinue function to make some routines for broadcasting message. Just one thing more : you said it's possible to retrieve every object of the same namespace. How ? Didn't find the way to do so.

About dereferecing, and object access, the thing is that a player in our game architecture as many variables. And following the advice of splitting huge objects into small one quick to (un)serialize, we've splitted the players into what we've called "managers" (maybe same sort of thing than your gameMode) which are dedicated to one part of the entity player management. (for exemple we have an itemManager for managin items, attributeManager for attributes and so on...). That's the starting part. And then each manager managed a few classes (also managedobject) for item, attributes, worldposition...

The thing is with such architecture, which (if I've well understood) seems to be right about data size optimization, we need to pass object through many methods and managers in order to user the wanted data.

For example if we put a new item which modify an attribute we need to pass the player, which gives the itemManager, the the item, which need to go back to the player to retrieve the attribute manager, and access to the wanted attribute (It's simplified here).
That's a lot of method with a lot of managedObject passed. So I'm scared that if we use direct typing everywhere we will have too much useless dereferencing.
In the other side, maybe we can improve performance by dereferencing every object only once...

So here come my new question, does it cost something to dereference a managedReference 2 times in the same task I always though there was some kind of caching system for reference during a task process, but I just want to be sure (if not, we surely have a problem with using this method).
I also wanted to know the cost of created new reference but I'm glad you answer no in your paper. ^^

By the way, would you (the darkstar team) be interested to know our game architecture using DarkStar ? It will surely help us about doing things right but maybe you're too busy to audit every darkstar base project.

(I allways dreamed about having one DarkStar engineer at my side during our game developpement helping us doing the good thing. ^^)

[Tim B]

The dereferencing question has been answered before (and in slightly contradictory ways).

Essentially:

1. Yes, the object is cached and only de-serialized once so the second managedreference.get is much faster than the first.

2. While it is much faster there is still some cost involved so it is still better to avoid it where possible.

[Tim B]

We've done a similar thing breaking our player objects up - although possibly not quite so much as you. (That may change as we get to profiling and optimising though).

Essentially all frequently accessed data is directly on the main java managed object.

Specific areas that might grow large and which are infrequently accessed are then broken out though. For example history (counts of number of matches played/won etc) are then in a separate managed object - as is the inventory/list of carried items/etc.
 

[enrodri86]

Much needed and totally appreciated piece of work. I can imagine that it will focus the community greatly to have a common framework, which can be extended with reusable custom code, maybe a whole engine could grow from this one day..

To be honest, I think a good public MMO engine is what is required to really push Darkstar forward, I know it's a bit fantasy land, but maybe a few years down the line the community could come up with some of the advanced features of the big boys, say HeroEngine or Big World. That would be proof to the naysayers that Darkstar can really cut it

The more I delve into game development, the less anamoured I am with the prospect of building and running a game, there's soo much responsibility that goes along with it, not to mention finding or creating good content. I think I'd rather research and develop Darkstar compatible modules, that would be fun, and Slipstream is the perfect centre point to tie in with.

Imagine a GUI development client, scripting, animation, terrain editing etc, maybe deployment system for new server code, a complete system would be amazing. Obviously there are so many ways to achieve this, but one standardized engine could be modded etc, it'd still cut down development time considerably.

I completely agree with you nokama, let me be the next to congratulate the authors for that great guide

A few months ago I was writting a project plan to start an online community which objective would be to build an open-source "standard" MMO engine, but I got discouraged... With the new extension system from the new PDS release, and your guide, I think the entry barriers are much lower!

I think I'll give that project another try, as I'm currently unemployed and bored...