Thanks, well, if you could convince Freescale to sponsor this effort, that would be the best :-D
But let's see first how this will work out :)

Konstantinos

I definitely think XRENDER and MySQL are the two best angles to go for here. I'm trying to extend the poll right now :)

_________________
Matt Sealey

I'm not sure if MySQL is the best candidate.
There are a few functions in MySQL which can be optimized to run a couple of times faster.
But these functions are rarely used.

If the main workload of a user is on such a function then such an optimization will highly feelable.
But I believe real hard to highly speed up the average database throughput for all users.

I dare say that MySQL is not the only application running on the system - it may be prudent to optimize MySQL such that it does not take up so much system time when you consider the traditional LAMP environment.

Most of the MySQL database engine is memory copies and string searches, hashing and referencing, all of which are vectorisable and show much improved performance. To have the database become not the main user of system processor cycles and to be more cooperative is good for any system.

At least that's my view on it from watching many servers over many years trying to handle heavy webserver loads, with the database being the bottleneck - not in IO, but in pulling huge queries, but in FULLTEXT searches etc. which are cached, a totally CPU-bound operation.

_________________
Matt Sealey

Matt, as you know I was looking into vectorizing MySQL some time ago.
You are correct that the general types of "searches" "compares" or "hashes" would be good target of vectorizing but MySQL operates completely with unaligned values. Vectorizing of compares has little or no gain if you work on short and unaligned values.

To get some performance out of this general task with Altivec you will need to do some major changes to the MySQL memory data structures to be aligned.

Another simple idea for speeding MySQL a little bit would be to switch to BIG ENDIAN on PPC . As you certainly know MySQL uses a "little Endian" storage format on all platform (on PPC too)

But as MySQL kind of dropped PowerPC as well supported platform anyway - I think you will not get any of the possible performance patches into the MySQL server anyway.

Well, in that case one could fork MySQL and develop a totally optimized version for AltiVec (Cell also?) systems. But is there enough commercial interest to warrant such a big project? Is there still time for a VectorSQL(TM) :-D ?

I agree the speed gain will be great on PowerPC systems, if one has the ability to make a complete optimized fork of MySQL. I know for a fact that if MySQL used a different hashing function internally than the one it does now, it could gain a significant boost just from that ( http://www.freevec.org/whitepapers/vector-hashing.pdf ). It would be even better if for example UMAC hashing was used (UMAC is directly vectorizable, though I have not worked on it yet).

Personally, I think the market for AltiVec is more mature now rather than dead, but that may only be wishful thinking. However, AltiVec is still strong (Power6 includes it, 970 includes it, even Cell includes it and would benefit from it, if there were no SPE support). It just needs the proper motive.

Right now, I'm more tending towards 3D software (Ogre3D, Blender3D and perhaps ODE as well) because I think it is a more fruitful market, but I might include a vectorized MySQL in that list as well, if there are enough people supporting such a motion. Funnily enough, I don't think X optimization will show a great benefit, and I'm not really in favour of this option. But that again may still change.

Konstantinos

Ogre3D uses OpenGL for rendering. And so XRender (mesa) should be optimized also. An optimized Ogre3D will probeably not gain for speed when mesa is not capeable to handle it fast enough.

Reading about databases make me wonder is sqlite already otimized for altivec. http://www.sqlite.org/cvstrac/wiki?p=CodeWarrior

I guess you haven't looked at the Ogre3D code. Myself, having actually looked at the Ogre3D code, I completely disagree. Most 3D game engines, are not just high level OpenGL wrappers. They provide facilities that are just not possible -or better, they are quite difficult- to achieve with just C and OpenGL. These facilities come at the cost of complex math calculations. For example, Ogre3D cooperates with ODE which is a physics simulation framework, providing Newton dynamics, fluid dynamics, etc on Ogre3D objects.

Also, Ogre3D itself does numerous calls to functions like Matrix4x4_mul or 1x4 (vector x matrix), is SIMD-ready (already provides SSE optimizations, which does offer significant speedup on x86 systems). Due to its nature, AltiVec would do more for PowerPC systems.

I hope I don't have to explain why these calculations will benefit from AltiVec/Cell. That is taken for granted, the important question is which software is more worth to optimize.

Konstantinos

The important question is, how many users are there?

It's not much use optimizing game engines nobody uses in games. For instance, a lot of people point out CrystalSpace as a great 3D game engine that could be optimized for this, that an the other, but can you name instantly one game that uses it? I will level the same thing at Ogre - it may be a really cool engine, but how many of these games using it (especially the commercial ones..) will actually download and run and take advantage of it?

I think, zero :)

Optimizing the underlying components would far better suit users - OpenGL (Mesa), OpenAL (after all audio tricks like 3D sound is all the same matrix math), XRender (well.. alpha blending is fun) and all the other basic system functionality on a system. After all, they are in far more use than maybe one noticable game running on maybe one game engine, not all people play games after all, but you can say these days everyone is using OpenGL.

_________________
Matt Sealey

http://www.ogre3d.org/index.php?set_alb ... _album.php

some look pretty commercial and very high quality to me...
Of course, noone will start buying PowerPC like crazy the moment Ogre3D is optimized, but it *will* make a point. And don't forget that we will probably make Cell optimizations as well, and that will indeed make a difference. I'll just mention that BulletPhysics (another Physics engine) already includes Cell optimizations.
You all keep mentioning Ogre, where it's only one of the choices mentioned. Blender for example is perhaps THE most popular FOSS renderer available and it doesn't even work properly on PowerPC (try to render sth complex and you will propably get a crash). Optimizing Blender -and fixing it in the process- will make PowerPC (G4/G5/Cell) a viable platform for rendering. Remember the High Density Blades? Here, you have a ready to run render farm that actually takes advantage of the hardware used. And you could actually sell it. My point is that 3D is very relevant right now. In any case, seeing that OpenGL is so much requested, I will at least look into it, if it's at all vectorizable and worth the effort.

Konstantinos

I too think that Blender is probably the only
viable choice in the poll. Databases doesn't run
on PowerPC. They run on x86, AMD64 and
UltraSPARC.

Large calculation clusters are built from a
performance and economy perspective. A friend
of mine works at a cluster development firm and
he said that x86 was hopeless when it comes to
calculations - POWER or Cell was the way to go.

BUT, since x86 are so cheap and after Microsoft's
entry with Microsoft Compute Cluster Server they
sell more and more of those.

If Blender can be optimized for PowerPC and if
Genesi could offer a complete rendering solution,
there could be a market. But it would probably
take multi-million investments to even get a
prototype running.

IBM doesn't seem to know that yet. And where I work we run Oracle on POWER machines.

Most of those games are for Windows only, and x86 only binary downloads..
So?

That only affects people who want to do 3D rendering and modelling on PowerPC.

I'm sorry but it is not a highly visible or very large market. Yes, giving it a few seconds optimization on rendering an image might be cool, but for the 10 people that might actually USE it rather than just saying you CAN use it..
Mesa already has SSE, MMX optimisations and SPARC ASM optimisations which make a huge difference. It is definitely vectorisable! Just nobody sat down and did any work with PPC yet.

I do think it's better to focus on the lower level system components before hitting on barely-used middleware.

_________________
Matt Sealey

Of course, noone bothers to port to PowerPC, because it's dead slow right now (no optimizations available). And of course, I don't expect that all companies will port to PowerPC because of AltiVec optimizations, but yes, I do expect a more extended usage of Ogre in low-spec games or applications.
Well, these 10 people might be just the persons who might be responsible for the hardware decisions in a render farm company, or similar. These are exactly the people that I'm after. Remember these are going to be proof of concept projects because I _hope_ that I will be able to support a business here.
If it's easy, I'll try to work on it. I saw some math routines that might be easily vectorizable. I'll let you know.

My point was more that noone bothers to port to Linux..

I think optimizing middleware is going to net you very few benefits, practically no marketable value on PowerPC for running game engines. If you can propose a port to Cell, that's one thing, then you have to convince a lot of Playstation 3 owners to install Linux, then run your game - this is quite unlikely to be a good sales pitch for your company :)

The low-level APIs are a good target as they optimize far more than just small smatterings of specialised software. If you want to get into specialised software, I don't think Blender is the right target.. it REALLY is a low-use case. I don't think having optimized Blender code really helps anyone.

_________________
Matt Sealey