[Ago]

It does not look like the original Avux idea is really taking off, (also due to the fact this forum is a bit slow). Therefore I am now proposing a new idea always called Avux (I like the name) even more ambitious than the original one:

Why don't we start a full blown project with a proper website, cvs tree, maillist etc with the aim to develop an open source STB multimedia ARCHITECTURE?


Something in the line of EOS ( http://www.avsforum.com/avs-vb/showt...hreadid=197595 ).


We talked about such a thing when this forum was started, then everybody went on his own to experiment with linux. My feeling is that instead of tapping the real power of linux we are replicating the "windows behaviour" trying to integrate different types of applications. This is leading to a lot of job duplication and dispersion of resources.


Furthermore I have not seen many open source projects with the aim to build a high-end multi-media ARCHITECTURE (not just a player, like mplayer or gstreamer, or pvr, like MythTV, or wrapper around an existing player, like Freevo). I might very well have missed some interesting project, if so please point it out here.


The focus should be on building AN ARCHITECTURE NOT AN APPLICATION where different bits and pieces can be easily swapped (eg you could choose what deinterlacing engine to use or what front end you prefer) and customised all the way from the processing of the raw AV stream (using pipelines?), to the use of additional filters (mixers, virtual channel extraction...), to the applications that can be included (browser, game emulators), to the media-files supported, to the codecs, to the menu customisation, to the possibility of using different types of display and control devices (OSD/handheld/Voice activated...). A fully flexible and customisable architecture, whose components can be plugged in like building blocks (hope you did like lego). As you see a significant step forward from the original Avux frontend project, not only a flexible frontend here. From such an architecture each one can get an stb exactly the way he wants it to work.


The focus should not be how to get htpc out of linux asap but how to get in the proper way and with as little compromises as possible. I think that given the code already available from other projects, 5-10 active programmers with the occasional support from others could get it done within a reasonable time.

 

[Ago]

Something like this

http://www.interact-tv.com/images/graphix/EOSblock.jpg


EOSâ„¢ APIs


ItvGUI: User Interface and Media Management designed to support entertainment based applications. UI Widgets, Flash 4/5, MPEG Video, Audio, Graphics, Truetype fonts, event management, remote control interface. It integrates all of the various media types to generate compelling user interfaces.


ItvXUL: XML based description language for describing itvgui based applications.

EOSâ„¢ Media Manager: Integrated database with media management support. SQL syntax, multiple connections for concurrent updates and management. Targeted at supporting a multi-user, multi-unit solution for the home.


ItvAppMgr: Application management including the application database, starting/stopping of applications, resource management and integration with the window manager (if the platform supports a window management system).

 

[Ago]

Note that the ItvAppMgr/ItvXUL/EOSMedia Manager are the equivalent of the "middle layer" I was describing in the original Avux project ( http://www.avsforum.com/avs-vb/showt...hreadid=186892 ). As you see my ideas on the frontend were not too lunatic after all...


As for the GUI it should obviously be easy to swap it, by making it only loosely linked to the middle layer with communication between the two performed via standardised xml files. Obviously the menu structure should be based on xml templates easily customisable by the users.


As for the way the raw A/V stream is processed at a low level I would deviate from the EOS and use instead something similar to gstreamer ( www.gstreamer.net/ ), see my article on pipelines ( http://www.avsforum.com/avs-vb/showt...hreadid=156908 ). Not just direct access to av libraries but only indirect access using an abstraction layer and a plug-in system where av libraries are seen as simple filters. Alternatively, instead of using av libraries via pipelines, external players (or other applications) might also be used instead of the libraries, where their behaviour is described in xml files accessed by itvXUL...


There is obviously a single nice multimedia DB, to take care of all meta-info and media library browsing, plus some EPG system, grabbers, rippers, etc....


Useless to say that it should be relatively easy to pack such system as a full distribution... But that should be only the latest phase of the project.

 

[Ago]

To start this project I really need the commitment of a few people willing to code, because I am extremely busy at the moment (work+phd+newborn baby daughter) and if you count on me alone to take things forward it will take forever. If anybody is (seriously) interested please let me know.

 

[Stone Free]

This sounds like a really interesting project to work on, but I wont really have time until about March, also I know nothing about Linux at the moment.


A good place to tout for programmers is at www.codeproject.com , or www.codeguru.com (if you can find somone without linuxphobia ). I would post in the Lounge at codeproject, I dont know exactly where for www.codeguru.com

 

[JohnAd]

Ago


Your ideas do seem sensible. I've been thinking similar things for sometime. The market only really supports monolothic front ends (like showshifter) at the moment and these often have problems with newer devices and networks etc. With a mish-mash of PC and consumer devices it is hard to see a way out without a clearly defined architecture for communication.


I've also been doing a bit of digging and the following might be of interest.


Universal Plug and Play see http://www.upnp.org is pretty much an architecture for doing what you describe and is xml and web server based, there is a specification on the website for AV devices and the spec seems pretty complete for the basic operations.


There is already linux code for this

http://upnp.sourceforge.net/


So your job may be as simple as creating the wrappers to allow control of apps via the upnp framework and the writers of the GUI only have to learn how to do upnp for which there is built in support on XP/ME/CE 3.0 and linux via the above code.


John

 

[Ago]

Thanks for the link John,


Very nice, but I am afraid things are slightly more complex, in fact from the look of it it seems that UPnP is mostly an XML standard for describing devices and services. That would be what in the EOS system is called ItvXUL. And in that respect UPnP is definitely something to consider.


Nevertherless, as you see it does not really cover all the possible operations, but only the application abstraction layer.


In particular for the description of media files and therefore also in the communication between Middle Layer and GUI other XML standards might be better suited. You do not want just to "turn on", "turn off" the gui, but you need to tell it what to display and the UPnP is not suited for it. Even though services and actions in Context_Menus (ie the set of controls/setttings for the player currently in use) could be well described using UPnP.


Also those standards do not really work for low-level communication, eg for A/V stream processing via pipelines, where again other standards are more suited for the purpose.


Plus there are many "fixed components" that need to be designed and coded, since not all the components should be "swappable" (eg multimedia DB) and used according to a plu-&-play system.


I hope someone will volunteer to help, simply designing the architecture on a piece of paper is a daunting task (let alone coding) and I need someone with some more experience than I have to help.

 

[JohnAd]

Ago


I'm sure you won't be converted overnight and the temptation to do your own thing is strong sometimes but given that you haven't really got any help yet it is probably worth starting from a higher base than you have been talking about.

Quote:

In particular for the description of media files and therefore also in the communication between Middle Layer and GUI other XML standards might be better suited. You do not want just to "turn on", "turn off" the gui, but you need to tell it what to display and the UPnP is not suited for it. Even though services and actions in Context_Menus (ie the set of controls/setttings for the player currently in use) could be well described using UPnP.

Don't know what you mean here.

Quote:

Also those standards do not really work for low-level communication, eg for A/V stream processing via pipelines, where again other standards are more suited for the purpose.

Take a closer look at the AVTransport proposal and you will see that the file to be played are specified using a urn allowing you to choose any streaming format including internally specified ones.

Quote:

Plus there are many "fixed components" that need to be designed and coded, since not all the components should be "swappable" (eg multimedia DB) and used according to a plu-&-play system.

Obviously but these components need to talk to the devices in a device independant way and there is no obvious reason why the fixed components couldn't present themselves as upnp components.

Quote:

I hope someone will volunteer to help, simply designing the architecture on a piece of paper is a daunting task (let alone coding) and I need someone with some more experience than I have to help.

Agreed, which is why I think tackling bite sized chunks is the way to go and not having to do loads of low level hacking around at the ip/http/soap protocol levels


John

 

[Ago]

John I see the point of your post, but again if you have a look at the EOS architecture you will note that UPnP is basically what they call ItvXUL. A very important piece, but there is also a lot of other stuff that cannot be included under the same hat... I agree that UPnP is an excellent starting point, but again, it is mostly an XML standard, your example on AVTransport for instance is simply a standard way to make a request to a transport in order to get a particular stream, so instead to use 10 different synthaxes for 10 different transports you use a single call procedure valid for all of them. This is basically what an abstraction layer does. Ie what UPnP does is "replacing" the interface or API of the components with a standardised one. There are obviously also other features, but this is one of the most important aspects. This is fine for sending command or parameters (without having to worry about what particular device you are using) but not for sending "content". For instance, in the communication between the Middle Layer and the GUI I need to send information about the content to be displayed, and not only about the interface, using UPnP for that purpose would be like using it in order to descirbe the video signal...

 

[JohnAd]

Ago


Sorry for any confusion. All I was suggesting was that you use UPnP for what it does well, for content streaming use whatever is suitable and there are plenty to choose from, just don't write a new method just for your architecture. The major chunk of your work is going to be getting the middle layer right as you say, what I was suggesting is that you could avoid a lot of design effort by assuming that the devices are found and controlled using upnp also you will pick up for "free" any other upnp devices that may appear. and you do not have to write documentation and tools for new developers as you can just point them off to existing stuff.


If you're serious about this then it will be a huge drain on your time and energy and the less time you spend on things like documentation and explaining to people how to use any new interface or protocols you dream up the more time you will have to actually make it work.


Microsoft is also building an archetechure to do almost exactly what you're suggesting called eHome initiative (I think) there were lots of presentations at WinHEC 2002 that you can download, obviously in that there is lots of secure path stuff but overall the aims are similar. Reusing a decent design makes as much sense as reusing code. I suggest you look through those presentations and see if there is anything you can pick up.


I still don't see the problem you seem to think exists with content metadata, that seems well covered by the av stuff.


Anyway good luck with your efforts, keep us posted


John

 

[Ago]

No need to be sorry, thanks a lot for the suggestion, it is in fact EXTREMELY useful and something I will most certainly consider. All I wanted to say is that unfortunately it is not enough by itself to solve all of the issue behind an stb architecture, but only some of those (namely ItvXUL which is a big chunk), and since there is still a lot of work to be done and my spare time is a rare occurrence I still hope that someone will move forward to help, particularly experienced linux users with knowledge about a/v-related-issues...

 

[Ago]

Well I am going to ask again. Simply designing (let alone coding) a flexible architecture is a huge task, one which I fear is beyond my knowledge and skills (as well as free time). Therefore if I am to start this project I really need some help here. Is anybody willing to contribute?


I would also be interested in general comments (also negative ones). Do you think this project is crap? Do you think it is superfluous? That it is too "big" to be accomplished? Thanks.

 

[m.a.n]

I'm no developer so this may be no use at all. But has anyone looked at this

http://www.networkmultimedia.org/


They seem to be building an mm framework for this kind of application



Mark

 

[Ago]

Woo very nice link, this is the closest project I have seen so far to what I had in mind. They even implemented a pipeline-based player!!! The GUI is still connected to the rest but it should be easy to separate it, a multimedia db is still missing, and the infrastructior is great for passing AV/multimedia content but not for other apps (like Upnp). Nevertheless, overall, the architecture seems to me to be VERY good, and the other bits and pieces could be easily added.

 

[Ago]

You should definitely read this

http://www.networkmultimedia.org/Sta.../mmbox2002.pdf


also this as a summary

http://www.networkmultimedia.org/Sta...roprix2002.pdf


We should contact those guys and have them to participate to the forum.

 

[orang55]

I can't help but asking if Muse.net is anything like what you're trying to do (because from a casual perspective, it certainly looks that way). It's win32 only for now, but looks a lot like what we're aiming to do. Go check it out (be warned, it doesnt work with wine)


Anyhow, sign me up. I'm sick of the lack of good free multimedia software for linux. I'll help with whatever I can. I don't have any programming skills, although I can assist in a website and testing.

 

[Ago]

Yes and no.


The aim is obviously to make such a STB a netowrk-client structure, but it should be much more flexible than Muse.net, allowing not only different types of media files (including games), not only custom menus with advanced search capabilities (ie sql based against a centralised db), not only different types of GUIs, but also different ways to play the files according to different sets of filters. Still I am waiting for someone else to help here. Even though there have been a lot of progress in many different areas and code is available, integrating differents bits and pieces is still a big tasks. I like a lot the idea of networkmultimedia of using a pipeline-based modular-player as the underlying engine. To this you have to add a single multimedia db, info grabbers (epg, cddb...), rippers, xml description of external modules, xml templates for menu structures and control panels...

 

[orang55]

agreed definitely. i was just using muse.net as an example of a framework layer (although the developers don't emphasize the distinction between the xml framework, the server, and client (as muse.net itself is nothing more than the framework)


Once again. Sign me up.


Should we create a sourceforge project page?

 

[Ago]

Quote:

Originally posted by orang55 Should we create a sourceforge project page?

I was inclined to do so, but I am ashamed that my knowledge (windows programmer with no real a/v experience) and spare time (close to zero) are not enough to start the project by myself. And to start something that ambitious, when you do not know EXACTLY what you are doing, nor you do have the time to finish it, would be simply a sign of arrogance and stupidity. That is why I was waiting for someone (with linux knowledge + low level av knowledge) to show up and give us some help both designing the architecture and coding...

 

[Ago]

Quote:

Originally posted by Ago I was inclined to do so, but I am ashamed that my knowledge (windows programmer with no real a/v experience) and spare time (close to zero) are not enough to start the project by myself. And to start something that ambitious, when you do not know EXACTLY what you are doing, nor you do have the time to finish it, would be simply a sign of arrogance and stupidity. That is why I was waiting for someone (with linux knowledge + low level av knowledge) to show up and give us some help both designing the architecture and coding...

After a bit of thought (I love to be publicly humiliated ) I decide to start a sourceforge project anyway, the name is obviously "avux" and this is the description I submitted:

Quote:

Avux is a multimedia set-top-box (STB) architecture, that allows different components (from players, to GUIs) to be used in order to create specific STB applications according to user preferences. The main aim is therefore flexibility, creating a middle layer that puts together different bits and pieces (GUI, multimedia DB, EPG, cddb, rippers, players, codecs, external apps), without being inherently linked to any of the components (so that they can be easily swapped). For instance the architecture should allow for different GUIs so that it can be controlled using either OSD+remote OR a palm-held pc OR voice activated system... Not just skins... Allow for menu structures according to user preferences (using XML files). Allow different players to be used in order to play a given file, according to specific parameters... Include or exclude additional components (web browser, emails, video conferencing)... The aim is NOT to create the underlying applications (eg media players), but to create an architecture that can use available ones in a seamless and customisable manner... The architecture will include 4 main components: 1) An abstraction layer, that describes available applications/services using XML (UPnP?). This creates wrappers around available applications. 2) A multimedia DB (MySql). 3) A middle layer (C++) that interprets low level user actions, queries the multimedia DB, controls the GUI and the available applications/services, according to user settings. User settings are specified in XML files, and they cover several aspects, from the menu structure to the way specific mime-types are played. 4) A swappable GUI linked only to the middle layer (communication with the middle layer via XML). Its output can be a video signal passed to an overlay engine and/or audio signal passed to an audio mixer and/or html app on webserver.

Do not expect much out of it, unless some guru will come to help... I do not see it as MY project but as cooperative effort.


IF the project is approved by sourceforge the first steps will be:


1) get some help for the administratrion of the project website (orang55 you have a job ), since I will not be able to monitor the website on a continuous basis.

2) set up chat and forum

3) start publishing a preliminary draft of the architecture (trying to say as few stupid things as possible...) as well as an introduction page about the project and its goals

4) get someone knowledgeble about a/v AND linux to actively participate

5) use the chat/forum to do some serious brain-storming on the architecture (could take months), consider what available code can be used, XML standards...

6) finalize documents on architecture

7) define roles, particularly project manager in charge of the cvs, maybe split the project in different components...

8) set up cvs tree

9) start coding (only AFTER all the above has been done...)