I think that unlike in the previous paradox which was "Which came first, the chicken or the egg?" which in fact was not actually a paradox, we should maybe coin a new paradox that's more accurate.

Which came first, the hardware design or the driver design?

Well, forget the paradox thing. Philosophy is a waste on this topic. We'll just discuss the problem more clearly.

Anyone who's ever openned up a development tool for designed chip logic will have come across VHDL at some point in time. VHDL, as opposed to being a programming language is called a Hardware Description Language.

When you develop a program using a programming language, you develop a series of code that runs in what could appear as a sequential order. You flow from one line of code to the next to the next.

When you design a chip using a hardware description language, you'd simple using a language to describe the behaviour of the logic gates on a chip. There is no particular flow unless your design specifically forces a flow. In reality, all you're really creating are blocks that have functions.

There are more advanced hardware description languages such as System C which blur the lines between VHDL and the C programming language, but it's still nothing more than a blur, it's still a hardware description language, it just handles some of the flow related issues for you.

Although I can write code in either type of language and can even make things work, the fact remains that it's not often you find anyone with skills exceeding mediocre in both paradigms. Programming and hardware engineering are both skills that take many years to reach an expert level in. In fact, in either circumstance, once you outgrown the basics, you've left the realm of engineering and moved into something more closely related to science.

Earlier on in PC computing, each peripheral needed to be designed with a great deal of care. The communication between the PC and the peripherals was typically painfully slow and unlike modern circumstances where even the cheapest PCs offer a PCIe x16 slot that theoretically communicates gigabytes per second, PCs rarely had busses capable of even controlling the simplest hardware in real time.

An excellent example of this would be the classic Commodore 64 which actually communicated with its floppy drive over a serial cable and the drive itself contained an entire computer itself. I even remember many people buying floppy drives that had more powerful processors than the machine itself.

In modern circumstances, we still do this on occassion. For example, PostScript which is a very popular format for communicating with corporate grade printers is a page description language that leaves it up to a processor in a printer to render graphics instead of the PC handling it. This means that much smaller images and graphics can be transmitted to the printer over the network where in the case of PCL (the most common alternative) the finished bitmaps are transmitted and consume much more network bandwidth. In my opinion, the real advantage to the postscript option is that the printer can choose to scale photographs to fill the page without caring about network bandwidth. This can really improve the experience for the user.

Well, all being said, the problem with a lot of modern hardware is waste and deadlines. These days, when a hardware designer announces they'll be making a new peripheral, it takes very little time before 10 other vendors release the same unit, probably selling it cheaper than the original vendor could make it for. And people will buy these cheaper models, and sadly the real difference is going to be driver quality more than anything else.

It takes time to develop a good driver. To make a great driver requires that the developers focus very heavily on error checking and handling. Drivers really should never crash, if they do, it's quite typical to crash the system too. This was actually the biggest reason for the blue screen of death we've all come to know and love.

It's also probably the best reason for stability in the Linux world which is that people using drivers on Linux will understand that they're written by some other guy that's just hacking around. People are happy to live with limited functionality long enough to get a more stable driver. In some cases, it can take years before a good one comes out. But, let me point out that many of the drivers released in a rush can crash just as ugly on Linux as on Windows. It's just that there's so much more cheap chinese crap supported by Windows than Linux. Linux users are typically better about buying from quality vendors, and if they buy cheap crap instead, they blame the hardware, not the operating system.

These days, if for example, I were to compete with the soon to be released optimus maximus keyboard which is a keyboard that has a single OLED screen on every key which is programmable by the system (or user), I would could come to market in 1/50th the time it took them to do it, ship a product that looks pretty similar, costs much much less, but doesn't work as good. These guys have used a huge amount of time and money to make their product and the thing that sucks is that cheap chinese knock-offs will put them out of business in under a year.

Their keyboard is better than what others can make, but all the other guys need to do is copy the concept. OLED is overkill, LCD is good enough. These days, there are hundreds of companies that can make black and white LCD panels that contain a controller on every key... and best of all they can do it REALLY quick.

Remember that digital watch your kid got in his happy meal? Well same thing... actually that watch is probably more complex technology.

Step 1 : Make a square LCD panel that fits into a key. Stick a tiny little microcontroller similar to the ones in a digital watch in the back of it. Make the resolution of the key 32x32 pixels. Good enough to display a black and white icon. Jam a tiny little white LED behind the LCD for back lighting.

Step 2 : Stick the key onto a switch, now you have a keyboard button. Wire all these buttons up the same way as on every other keyboard made since 1981.

Step 3 : Run a four wire bus to every single key, just connect them all in parallel. Power, Ground, Data and Clock. The keys are output only and don't need to feed back to the system. Using data and clock makes it so that every processor ever made can be used to program the keys. Adding a fifth line to allow a key to identify itself allows for easier manufacturing. Then a startup program can be used to make each key blink.

Step 4: Add a cheap ass microcontroller to the unit. The faster the controller, the faster the keys can change their faces. Connect the contoller to the PC using USB.

Step 5 : Write a crappy user mode program for Windows that sends the bitmaps for the keys to the keyboard when someone clicks a task tray icon. Add a little feature allowing them to make their own keys.

Entire time from concept to market on a product like this can actually be done in about a month. Best part is, at about $1 per key + $50 for additional electronics, you can probably buy it for $200.

After you've sold a bunch of them, you can kill off the business so you can dump support, and then design a better model. Before you know it, you'll be selling a high resolution color version and selling it for $250 a pop. But on that version, hopefully you've taken the time to make good drivers.

When I was in school and first learned about Darwin's theory of evolution, I was maybe 10 or 11 years old. Of course at that age, the teacher I had, an elementary school teacher, required to teach all subjects, not just science was hardly qualified to provide anything more detailed than a basic introduction to the topic. In fact, I remember that she was in fact a religious person and the theory of evolution contradicted her personal beliefs and if she weren't required as part of the science curriculum to discuss the topic, then she would never have taught it in the first place.

Well, I remember after hearing this theory, my first question was "Then the egg came first?" at which the teacher reminded me that we were studying science and not philosphy at the moment.

I recently heard for the first time in years the paradoxal question of "Which came first, the chicken or the egg?" at which point I explained to my collegue that the argument was gibberish since what they're actually asking is "Are you religious or not?" and it didn't pertain.

Wow, sometimes I'm truly stupid since, as you might have noticed, I often rant and rave. When asked to justify my statement, I offered to do it later, but then found that the people around me were not going to just let that statement lay still until I explained it more clearly.

I explained they should read the wikipedia entry on the topic, of which I was sure there would be one, but frankly, now that I've read it myself, I realize that the people that don't understand what I had said aren't likely to understand the article either. That in my opinion is the bitch of Wikipedia. I had even considered making a dumbed down section on the Wikipedia article, but realized that the article, while well researched was poorly written and should just be left alone until someone more sophistically educated than I were to take the opportunity to rewrite it.

Ok, so here's the answer to the real question when asking "Which came first, the chicken or the egg". As you know I'm not religious, so you'll know which answer I believe.

For the religious, the answer is simple. The chicken came first. The reason for this is that relgions that accept the book of Genesis will clearly agree that God created all the creatures on the planet as they are and therefore, he must have created at least two chickens that further reproduced by laying eggs. Simple as that, done, no possible room for argument. The chicken came first.

For the evolutionists that answer is simple but has a twist.

If the closest ancestor to the chicken which of course was not a chicken laid an egg of which a chicken was born, then you'd have to decide, is the egg which holds the chicken a chicken egg, or was the first egg laid by a chicken a chicken egg. It's purely an issue of deciding whether the type of egg is defined by the animal that laid it or the animal within it.

I believe this issue is also easy to resolve since I prefer the term "Chicken's egg" over "Chicken egg". Since the chicken occupying the egg does in fact own the egg. After all, it's not as if someone else is going to evict the chicken and move in instead.

So the answer for an evolutionist is also quite simple, the egg came first since the closest ancestor to the chicken laid the egg containing a chicken and therefore produced the first "Chicken's Egg".

I have worked in mobile telephone development and deployment with nearly all the major vendors of handsets. In fact, I've had piles of project managers and developers from nearly every firm babysitting my projects for years. It's incredibly annoying. To this day, after working on 15 major smart phone development projects, I to this day have no idea what these vendors are thinking, or more to the point if they're thinking. With the exception of Sony Ericsson who at least takes the time to admit that they don't know everything, the rest of the vendors just make blunder after blunder which causes their phones to be released late or they simply miss their release windows and move development to the next project.

I have suffered through painful toolchain disasters brought on by Symbian. I have suffered through disgusting windowing toolkit reinventions by Motorola. I have anguished through GUI redesign problems by UIQ. I have drowned in the chaos of Ericsson botching the initial reference design of bluetooth by merging it with a project too big to chew in a limited time. I can even tell you of failed attempt after failed attempt of creating GSM/GPRS stacks by intern students at most vendors. I can go on and on and on.

So let me start getting to a point here. Nokia could have done something great back in the day when they adopted Symbian, but they failed miserably because they could have started from the ground up and made something great, but instead they bought into a lie.

See, Symbian is a market share holder not because of great technology, it is a market share holder because of great marketing. This is a problem for Nokia now since iPhone is actually almost great technology with awesome marketing behind it. Apple can in fact take this market by storm and go straight for monopolization by playing the underdog card until everyone realizes it's too late. The technology has started off expensive, but Apple will eventually release mini and nano versions of the iPhone which cost less and still provide a great deal of functionality.

What Apple did was simple. They made use of technology that they developed over a period of several years, focussed on a single telephone product, bought the right parts from the right vendors, invented better pieces where they were needed and put it in a pretty package. They spent 6 years building a the strongest media delivery service ever seen.

So who could possibly compete against that? Well, I'll tell you, there are a few strong possiblities.

Microsoft could if they would take hardware design serious for once. They have the talent, but they prefer to waste their money in marketting as opposed to developing a product that could nearly sell itself.

Amazon could do it since they could buy most of the technology firms they need and then merge their content business with it.

Walmart might even be able to pull it off if they were in fact interested in this.

Sony, not Sony Ericsson might be able to do it, but only if they bite the bullet and actually recognize that consumers love the content pricing model of iTunes Music Store.

Creative could partner with Microsoft, Walmart or Amazon and pull it off. But they have to be willing to commit to a single model that rocks as opposed to multiple generations of models that just don't pull it off.

Let me focus on Creative for a while since my brainstorming as brought me to the only competitor that seems to have been able to feed themselves when competing against iPod.

Creative has the hardware engineering talent, but they lack the asthetic design talent. Not once has Creative made a single device that catches the eye like the iPod does. Creative misses the mark because they think that iPod is about features when the first attraction for most can be boiled down to "Oooh... Shiny!!!". The iPod catches the consumers eye by simply looking cool. If you want to get the consumer attention, it's about giving them something flashy and pretty.

Usability has been a huge problem for Creative. They experimented with all kinds of different ways of sticking songs onto phones, but the truth is, they have never come close. From trying tools like MusicMatch to Winamp plugins, to custome made crapware, they have always missed the mark. The reason is simple, they don't just KISS. iTunes is a force to be reckoned with since any idiot can use it. It doesn't give the users any of the functionality of the more powerful programs (including Windows Media Player or Zune), but it does make it so simple that anyone could use it. The goal isn't to go after the power user... if the technology is stable and solid, the power user will produce their own solution. The goal is to get the general consumer.

Creative needs to give their software away for free, make it a strong alternative to iTunes. iTunes doesn't ship on many computers, instead it's either installed via CD from an iPod box or it's downloaded by a ton of users. Make a great player application and just give it away for free. Make it work on Windows, Mac, AND Linux, and most importantly, DO NOT WRITE IT IN JAVA. Write the application in C++ and Qt. Write the drivers independantly for each platform. Provide a customized file system so that when a user browses the device, the system files do not appear to the user. It will obviously be hacked soon enough, but for the basic user, there's nothing to be confused by.

Now, here's the next bit. If the goal is to compete with iPhone, a PDA is a must. Alot of solutions exist for this, but the fact is, beyond an OS kernel, there's no such thing as a good UI. So here's what I'd do.

Install Windows Mobile, yes Windows Mobile. Forget the GUI provided by the operating system, it's crap and it looks too much like a PC. Think phone. Nokia became successful by shipping a crappy operating system with a stable communications controller and a simple single Icon per screen selection. Make this one simple too. Here's how.

Go to Opera Software, insist that they alter their OpenGL based rendering to support mobile DirectX instead, their developers are very bright and can do this with little effort.

Hire a top notch web development team that can develop "Web 2.0" applications. The people you're looking for are artists that make the best Windows Gadgets and Opera Widgets. These people are bright and artistic and do a great job.

Use the Opera client side ecmascript engine to provide "objects" to the telephone functions. I'm quite sure they already have many of them available.

Go to Adobe and pay the license to the Flash player. Macromedia was vicious about this, but I hear Adobe is more friendly. Make sure to get Acrobat as well... AND GET CODE TO PRODUCE PDF AS WELL.

Develop into the desktop application the ability to sync contacts between the phone and Outlook, Vista Mail/Calendar, Lotus Notes, and Thunderbird/Sunbird, I guess Apple Mail and Address Book and iCal would help as well. Do no reinvent the wheel, use what's there.

Java might be necessary for service providers, but thanks to Web 2.0, it's probably not that important.

Now that the phone has a basic design, focus on an SDK. The phone must be easy to develop applets for. Make it so that making a new applet can be done by anyone. Every single object needs to be clearly documented and the entire installation proceedure needs to be flawless and well documented. People should not be confused as to how this is done.

The most important thing that needs to be understood at this point is, once this system is made, only evolution is allowed. User testing labs should be paid for in Asian, European, and American markets. Professional services should be used to make sure that the UI is easy to use and more importantly, when placed on a shelf with 10 other phones for the user to choose from, they will choose this one off the shelf to take home with them for free.

Once the phone is designed, the packaging and styling is sorted out, a marketting campaign needs to be launch that doesn't focus on making the phone better than iPhone, but instead should market it based on its user friendliness, openness and other merits. The consumer should be interested in seeing it because it is affordable and won't make them feel stupid.

Work out deals with Slingbox or Tivo for copying directly to the device. Make it clear that the user isn't locked into content from your store. Make it clear that the user has the freedom to choose.

Well, I can go on for a while, but frankly, I am quite confident that Microsoft, HTC, Creative, Nokia and Ericsson which just continue on bumbling down the road trying over and over again to clobber the mammoth and they just won't invest what it takes to actually compete against Apple which is a company that manages to maintain approximately 10% of the world PC market, 75% of the world music player market and are quickly making inroads into other markets. These companies will just thing that iPhone is a passing fad up until Apple releases the consumer priced phones which just pound them senseless.

I would like to point out the most important reason Apple which succeed if noone else steps up, they develop the technology in harmony. They build an environment where development teams produce all the components with the hope of making something truly cool. They don't waste time blaming everyone else for failures, they focus on what matters, the product!

P.S. - If anyone at Nokia is listening, I'm willing to gamble a chunk of my salary on the fact that iPhone will crush anything Symbian based. Not because I don't like Symbian (which I don't), but because Apple recognized that Symbian wasn't an option and understood they had to start from the bottom up. Symbian was obsoleted the day users wanted more than just a PIM.

Tags: creative labs, ericsson, htc, iphone, ipod, microsoft, nokia, pim, symbian

Life can be amusing at times. When I moved to Norway back in '98, the value of the dollar was growing rapidly and since I was employed in California, my $100,000 a year didn't really get me that far. I really hadn't become used to the value of items here in Norway yet and I regularly found myself bleeding my bank account dry. I'm happy to say, with age, I've learned to budget. You have to here in Norway, it's common here to be paid monthly instead of weekly or bi-weekly. This is a great thing for me since I always had problems making rent when I was a teenager since my weekly salary wasn't nearly enough to cover it and I never had money left over from the previous week.

Well, here's the deal, the value of the dollar since Bush came to office has dropped so drastically, that in almost all circumstances, it's cheaper to buy products from the US, ship them here and pay tax on it. In fact, the value of the dollar is so lousy, European travel spots are giving heavy discounts to Americans because they can't afford to travel here anymore.

Well, the high mighty dollar is in the toilet, so this means something big for Americans. If you're a consultant, get your passport NOW! Abuse short-term work laws. See, most European countries will let American's come over and reside for up to 3 months per year in a European country without a work visa as long as the European firm you're working for is paying an American company for your services. So buy a Delaware shelf corp (about $1000) file for a name change and get out to Europe to start contracting.

See, Europe right now has the same problems as the states in regards to hiring qualified workers for technical positions. In fact, although I don't have the exact statistics, I've heard rumors of tech-sector unemployment rates of 1% or less in Norway. Useless people that lie their asses off to get jobs are getting employed since companies are forced to hire people "willing to try" since qualified workers can't be found.

To give you a feel for pay out here, the girl that cleans my house gets $16 an hour + tax. She's a Polish imagrant that barely speaks Norwegian or English. So she even lacks the minimum communication skills you'd expect from a worker.

Working as a waiter/waitress at TGI Fridays pays about $15/hour + tips. So, figure that $15/hr (without overtime) is about $30,000 a year, and $5-$10 tips are typical. So, if you serve an average of 4 tables per hour, that's an extra $40,000-$80,000 a year. Of course, maybe you only average 2 tables an hour because of dead times or you have to share them with the bartender, so that's $20K-$40K in tips a year. That's a salary of $50,000 to $70,000 a year to be a server. And the two TGI Fridays in Oslo are located in the hottest tourist areas, so English speaking business travellers that are accustomed to 10%-15% tipping (English/American) will shell out an average of $15 per tip since even the cheapest meals for two people with two light drinks will typically run $100 here.

It's quite humorous to think that the divide between service staff and professionals here is so small. A typical University graduate with a degree in computer science will usually start at $50-$55K a year. Within 5 years, with the ability to sell themselves, they can typically earn $95K a year. Consultants here regularly charge about $75-$300 an hour depending on the skill set. A Cisco Certified CCIE will typically bill about $300/hr. A VMWare VI3 certified can easily make $200,000 a year (with travel within Scandinavia), in fact, I've been tempted to take the test and take a job I know about, but I can't live without seeing my kids every day.

Business jobs are also very strong here. Norway doesn't treat business as a university curriculum. They use separate business schools for that. Sadly, unless a business student spends a few years outside of the country, they will leave the school understanding little more than designing a business plan and social networking. Because of this university educated business grads are in high demand. This includes marketting as well.

The problem is, the business schools here almost entirely lack general education. Meaning the only math is probably statistical or accounting. So, business people here are at the mercy of outside consultants to solve simple geometrical problems related to calculating usable office space. They might also learn statistics, but they're more likely to hire a math grad for that job. Out of all the business school grads I've ever questioned on the topic, less than 10% could use math to justify predictions they have made based on graphs. Instead, they just point and say, see that line, it's going up. Or see that line, it's going down. I have very little respect for statistics related to business since the data-sets used to calculate the statistics are generally mined to provide justification to an opinion.

So, if you're a business grad from Columbia or better, get out here. Your knowledge of capitolism can easily land you a job with options in a company that focusses on hyping their shares. An excellent example is Opticom. A miserable company that could not under any circumstance make the product they claimed to be making, but they hyped the share and it showed 1000% (yes 3 zeros) growth in a short term. Most employees could never cash in on more than 25% of their options, but that was still 25%. The business-folk in the company pretty much all got rich. These companies come and go quickly here. The reason for their market success is simple. The market here is tiny and even hobby day traders can have a huge impact on share values. The volumes of trading is just so small that $100,000 can cause a major upward trend. If you're trading short, it's no challenge, just throw 100,000 shares of whatever onto the market and force a 5 point decline. Happens all the time. The Norwegian market shows +10%-+20% performers every day in the top 5. The declines are equally drastic. Oh.. and the Norwegian market has almost nothing to do with company performance, it's pretty much entirely controlled by day traders that lack the ability to comprehend things like the difference between turn-over and gains.

So, in short, if you're an American, get your consulting rolling here today. If you're a really lucky one, get a Norwegian job and work from the states. You'll feel rich.

As for Norwegians, this is the ideal moment to invest in US dollars. At 5.9kr to the dollar, it has to go up soon, after all, if it goes down any further, it'll bring the world market with it. The dollar right now is about as strong as pre-war german marks. If the dollar doesn't start going up, Americans will be rolling wheel-barrows of dollars to the market to buy a loaf of bread.

Tags: international consulting, norwegian economy, us dollar, work overseas

Don't buy it! Not Vista compatible!

That's right, don't buy it. I bought one about a year ago. It was a great idea, buy a Lacie gigabit ethernet 1U 1-Terabyte disk. How can you go wrong with Lacie?

Well here's the deal, they advertise a few things, but I have a pet peeve.

If you're going to sell a disk and advertise that it's 1-Gigabit network compatible, it takes more than just plugging a 1-gigabit Ehternet adapter into the system board to make it work right. 1-Gigabit means there's a theoretically limit of approximately 125megabytes per second transfer rates. Now in reality after protocol overhead and such, at least 80megabytes per second should be realistic. So that's not hard. The machine has four, count them four hard drives capable of 50Megabytes per second installed in it. For sequential file access, 80Megabytes/Second should be the easiest thing in the world.

Well when the device is connected to my massive wx8400 (capable of spamming at 160Megabytes per second using teamed gigabit ethernet) and through a HP Procurve 5400zl monster switch with ungodly performance capacity, the transfer rate to the Lacie disk peaks at 13.3 megs per second.

That's right! 13.3 Megabytes per second. That's approximately 1/6th the capacity of the weakest link in the chain.

Why is the performance so low you ask? Well it's easy, first of all, the unit ships with a Via Technologies 800Mhz mini-itx motherboard. It also ships with 256 Megs of RAM. I've upgraded to 1 gig of RAM and am now investigating replacing the system board altogether with a Pentium-M 2Ghz Motherboard. It's just a waste to try and make this thing perform otherwise.

Also, the unit ships with Windows XP embedded. An operating system that suffers all the security problems of Windows XP but lacks the ability to be upgraded through Windows Update. So, as security holes come up, the box just stays susceptible. Well how are you supposed to update Windows XP Embedded you ask? That's easy, you can either add automatic updating support or you can send out regular updates to the box.

Lacie of course is a company with a long history of support and quality right? Well, apparently they got themselves in too deep when they chose XP embedded as a solution.

So why isn't it Vista compatible? Well a while back (XP service pak 2 I believe) Microsoft altered their authentication method to make it more secure. In fact, they disabled supporting machines that used a version of SMB that was too early unless you made a registry change to allow weaker security. Well Vista is shipped with this by default and turning it off is a security risk. So Lacie should have released a patch by now.

Well, I've had the unit long enough to let the warranty expire and there has never been even one single update to the unit and I've stopped holding my breath that I'll ever receive one.

So, I love the case design, I'll replace the drives with terabyte S-ATA soon and I'll replace the motherboard and fans as well. Meaning I'll keep the case and power supply and dump the rest.

Lacie has proven yet again, the only thing they're actually good at is making a pretty box.

Tags: lacie, vista compatibility., windows xp embedded

Ok, I buy a bunch of high end solutions for video processing. This is no secret. But recently, when I called my local dealer to buy a second Canopus ADVC1000 for another capture machine, he told me that it would take a few weeks, but he could loan me the ADVC3000 which only costs twice as much in the mean time if I tell him how I like it.

Well here's the deal, it's been here for about a month and we've been using it so much that it's actually begging us for a little down-time :) The ADVC3000 is a champion. So I called the salesperson earlier and informed him that he can keep his stinking ADVC1000 and we'll take the 3000 instead.

So why do I like this unit so much? I'll be straight, there is at least one bug driving me bizerk, but the box just works. We're not an HD shop here yet since there's just no real demand for it yet. We have the means to support HDV if necessary, but I think I've run a single tape in the deck this past year. We do almost all SD.

At my disposal, I have two systems sporting the BlackMagic Extreme, one with a AJA Xena LH, another system running the Canopus ADVC1000 and now another running the ADVC3000. I obviously realize that for full frame 10-bit uncompressed, the Canopus units will never be satisfactory, but we do tons of work here on creating corporate DVDs and preparing film for low-bandwidth broadcast using DVB over UDP Multicast.

We only use CinemaCraft Encoder here. I would die before switching away from it, there's just nothing that even comes close in the hardware or software arena. It's just the best MPEG-2 encoder out there and I know they have MPEG-4 on their real-time systems now, but it's out of my price range just yet.

So let's talk about what makes an SDI to DV media converter a better option than the more expensive 10-bit uncompressed solutions.

1) Drivers are never a problem. I never need to screw around looking for a driver to support the device since it appears as a DV camera or deck in every program. The only place I've had any problems so far is that if I am working on a crap video and want to real-time encode using Ahead Nero, Nero's deck control can't handle long clips. Also MainConcept Encoder is laughable at best regarding real time capture.

2) For the most part, 10-bit is wasted on encoding MPEG since you really shouldn't use 10-bit DCT coefficients on encoding for DVD. The reason for this is that the MPEG-2 Video spec ISO/IEC 13818-2 requires that the quant_matrix_extension() is honored to allow for 10-bit DCT quantiser and although theoretically this extension should always be honored, the cheaper chipsets rarely do. So 10-bit MPEG-2 encoding is typically just a waste of time. Also, the color depth of most modern LCD and plasma displays don't do much to support deep color. It's just best to leave 10-bit and greater depths for HD-DVD and BluRay. So the Canopus ADVC 3000 does just fine for the color depth requirements.

3) You might say that DV kills quality and for the most part, you'd be right. DV is a color banding creator. After all, if you've ever dubbed a Digibeta to a DVCam, you'll have learned that banding is a fact of life for DV. Well, here's the best part, it's not really a problem with the Canopus since the Canopus encodes all frames as I frames and the 10% color loss typical of DV is minimized to something closer to 4% in my experience, though I intend to run formal mathematical tests when I get time. Full I-Frame DV is really just great quality especially when the unit bitrate is set to S400Mbit instead of S100Mbit. And I'm only guessing, but it feels like the quality of the video coming out of the ADVC3000 is higher than from my DAC-5 which should do the same thing.

4) Sound channels, well in the case of DV, there are two possible configurations. Encode two channels at full quality 48Khz/16Bit or encode 4 channels at 32Khz/12Bit. This is the killer of DV. I mean seriously, full frame D1 uncompressed only requires 20MBytes per second to transmit. DV could easily add in four or eight channels of uncompressed audio, but this sadly isn't the case. The Canopus honors the settings for DV and because of that, if we want more than 2 channels of audio, we either need to pass it through an M-Audio firewire box, or we need to use the AJA LH to capture all embedded SDI audio channels. Well, that's fine anyway since for the most part, channels 3&4 are rarely used for anything other than dubbing audio (music and sound channels)

5) Performance. I have a hard drive array with twelve 500Gig hard drives connected through a high end Adaptec controller. In theory, I can handle 12x60Megs per second throughput, but I round to 50Megs because I don't trust theory. That means I can, with full stability transfer 600Megs per second to the PC. So, that would be approximately twelve time as fast as full frame 10-bit requires. I have a machine with 8 2Ghz Xeon cores, why should I ever notice that it's full frame? Well I am guessing it's CODEC related. But if I work with high quality DV captures, then CinemaCraft runs at 8x or better on this machine. So, if I capture 2 full hours of video and then want to compress it, it takes me approximately 15 minutes per pass to encode. On video I care about, I run a minimum of 4 passes and regularly run as many as 15 passes on low bitrate video. DV makes my life SOOO much easier. As for quality, I get nothing but compliments from the customers.

6) Mobilty. I regularly need to capture video to my laptop, I have a Toshiba Qosmio G30-219 with two of the slowest hard drives ever made. They're big alright, but a video notebook should have 7200RPM or at least 5400RPM, this is 4200RPM and because of that I need lower bitrates. Well, here's the thing, I can plug right into the Canopus ADVC3000 and capture directly to my second hard drive and never worry about dropping frames.

Well, I hope this helps anyone thinking of making the jump to the ADVC3000, see the thing is, if you're a math head like me, meaning someone who calculates every bit lossed and complains about every single one of them, then an SDI-DV converter is typically a sinful thing. But I've learned over time that sometimes you have to just settle for "Good Enough" and while other solutions might be "Good Enough", the ADVC3000 is just as good as you're ever gonna get in the DV world.

I'm really hoping to get a chance to evaluate the Canopus HDM1 since I'm currently looking for a realtime MPEG capture solution. Sadly, I see this unit being too little too late. As far as I can tell, the box only encodes MPEG-2 (although high-profile) and even though ATSC dictates MPEG-2, there's very little use for this when MPEG-4 is already here. Also, Norway, where I live has opted for MPEG-4 on the DVB-T network over the MPEG-2 ATSC style of terrestrial broadcast.

Soon I intend to check out the CinemaCraft MPEG-4 H.264 encoder unit if I can get my hands on one. Since soon I'll be delivering high-definition to my key customer, it's important for me to test solutions. For the moment, I'm torn between Ateme and x264 for encoding and frankly, I think they both suck quite bad. Mainconcept just doesn't make good enough encoders (stability wise) for me. I find that it's difficult at best to predict the quality of results you'll receive from them.

Tags: advc3000, aja, blackmagic, canopus, cinemacraft, h.264, mpeg-2, mpeg-4

In my many attempts to extract just the main movie from a DVD, I believe I finally understand the structure well enough to comment on it.

The structure of a DVD is somewhat silly since the movies exist on a 4.7-8.4 gigabyte disc. On a double sided disc, the amount of data it can hold doubles, but still, they insist on using a binary format for storing the data structures of the disc. I guess someone somewhere considered this to be part of the copy protection. After all, if you make it confusing enough maybe people won't go out of their way to decode it... RIGHT!

Ok, I've spent a while coming up with a solution to be able to parse the IFO structure of the DVD. Now the kicker of it is, there are already libraries out there for the task such as libdvdread, but they're not particularly useful since the library is coded in C and is more or less a big-ass glop of pointer madness. Instead of making a nice, clean, documented, usable library, they simply implemented all the code necessary to read the IFO. What's worse is that DVDs use acronyms for describing everything. They actually use acronyms when clear names would be shorter, it's just madness. And the other libraries for reading the files I've come across insist on honoring the acronyms instead of using an extra 5 seconds to type the name out.

The only other library I know of which is somewhat usable is written by the author of SubtitleCreator, and while his implementation is more clear, it's extremely limited and only implements the parts of the IFO which are interesting to the author.

So I've been working on implementing a full alternative to libdvdread in C#. Yes, I chose C# because it's very easy to code in (Intellisense does most of the work) and C#, while a cheesy for real coding is actually easier to read and document than C++ (my prefered language). Oh, let's not forget that I'm slowly moving away from C++ because it just costs too much to buy Qt from Trolltech every year.

I haven't even considered writing the DeCSS component into my code since programs like AnyDVD do the job better and it writing a CSS implementation without a license is still only questionably legal outside of countries like Finland.

Well here's the reason I went through all the trouble. I rip about 100 DVDs a month for broadcast purposes and none of the "integrated" solutions actually prepare a CableLabs compliant, high quality, constant bitrate transport stream with DVB subtitles. The closest thing to it would be from AnyStream, but the problem is, they don't do subtitles. So what I've done is to use a combination of :

- DVD Decryptor to rip the DVD to hard drive
- DGIndex to open the DVD movie and prepare a frame server
- AVISynth to frame server to CinemaCraft
- CinemaCraft encoder to reencode the video multipass CBR since simply requantizing won't work on these streams that are typically barely MPEG compliant to begin with.
- BeSweet to extract the AC-3 Audio to 5.1 Channel wave files and stereo wave files
- Sony Vegas 7 to reencode the audio to conformant AC-3 and MPEG-1 Layer II streams
- My own subtitle ripper to extract the subtitles from DVDs
- My own subtitle converter to prepare the subtitles as DVB compliant streams
- My own MPEG-2 Transport Stream multiplexer since Manzanita just costs too much and I'd have to write a multiplexer anyway to incorporate the subtitle streams. And the other multiplexers are either slow hardware or just crap anyway.

So what I'm doing now is to make a program that reads the IFO files from the DVD, present the streams to the user, allow the user to select the main movie (clipping it if necessary), then prepare the MPEG-2, AC-3, DVB Subtitle, and MPEG-1 Layer II audio streams. Then the user will be presented with information relating to the streams and will be allowed to select the PIDs and included streams.

The next step of the project will be to make a server repository for streams that have been previously prepared. Oh and since I've written a full DVD parser, I'm intended to allow the code to export a full DVD studio project from a premade DVD so that my editors at the office can make small changes and make new DVDs.

Oh as a last bit of my trademarked negativity, the DVD virtual machine is a heap of crap since the op codes are unnecessarily weak and instead of providing a proper relative jump instruction, it appears to be locked into absolute line number references. This is just slop.

Tags: ac-3, decss, dvd, ifo, mpeg-2

Well, in this posting, I intend to ramble and discuss one of my greatest pet-peeves. Memory management on embedded devices. See, I worked on developing embedded web browsers for a long time and the fact is, the more I think of it, the more I realize that a program like a web browser on an embedded device such as a telephone should almost always be written in virtual machine environment. HUH?!?! That's right, I believe that programs with intense memory allocation requirements should be done in virtual machine environments. And I'll go on to why.

See the best web browsers out there are written in C++. Opera is, Firefox is, IE is, and even the lame excuse called Safari/Konquerer is (sorry, I just don't like Konquerer, code is beautiful, browser sucks). They're written in C++ since it's a language that works on all platforms and has all the great features of object orientation. We can argue all day long over what the best language is, but C++ is proven over time and anyone with a solid understanding of data structures can code it really well. If you can't handle data structures, use some other language that takes care of it for you.

A web browser is in fact the worst thing you can ever install on a platform. It has to be fast, it has to feel smooth and people use it more than any other application. A browser has to be able to parse, manipulate and display pages and scripts written by millions of poor coders around the world. A web browser that implements all the primary standards (HTML, DOM, XML, CSS, Ecmascript) needs to be able to parse all the elements into a relatively standardized structure (DOM) which regularly contains thousands and in terrible cases millions of individual elements.

Unlike other applications, a web browser will make millions of memory allocations of all different sizes ranging from a few bytes for a short text element to a hundred megs for a REALLY high resolution image. The order in which the allocations occur is not defined by the browser developer as much as the page designer and the page viewer's use of the page. So in short, from a memory allocation perspective, a web browser is the most evil type of application ever invented.

So how should a system deal with these special requirements. I've worked on telephone that shipped with a total of 4 megs of Flash and 6 megs of RAM (weird number, but true). I've also worked on telephones that ship with a gig of Flash and a gig of RAM. This was much nicer, but still problematic. See, when you're working on a telephone, it is imperitive that the applications are rock solid stable. See, except in cases like Windows based phones that behave more like a PC and less like a phone, applications don't usually crash nicely on telephones. In the case of Symbian, most of the time, using up all the memory on the phone will cause the phone to crash.

Symbian is famous through the embedded device industry as being an operating system that reinvents every wheel, each time they experiment with adding more sides or different shaped grooves. They refuse to make it round like the next guy since that would be intelligent. They reinvent the graphics system... with every minor release. They reinvented the storage system, they reinvent the program load system. Hell, they don't even like standard executable file types, they alter them after they're done linking them. But one thing they tried to do, with amazing failure was to reinvent the memory allocation system.

Symbian tried to implement a whole new memory architecture using incredibly well thought out plans with an implementation that is equally as bad as the idea was good. First off, let me say that their clean-up stack solution to memory handling was their worst idea ever. I like exceptions almost as much as the next guy, but if you make the entire defining point of your operating system development experience a hack to clean up after programmers for them, things are bound to go wrong. The result is, programmers spend 5 times longer trying to use the clean-up stack than programming their systems to being with.

Symbian tried to implement a memory allocation system so that it bring the features of the MMU forward to the application developer and make it possible for them to use multiple allocation pools of memory quite easily. It was an incredibly good idea so that applications such as web browsers would have a much easier time releasing memory to other applications in need than to deal with stack fragmentation issues. The problem is they didn't put enough focus on the design of a signalling mechanism to inform applications to release the memory at the right time. Also, it appears that it was an asycronous operation. So if app A needed memory, it would try to allocate it. If app B could release some memory, it was informed until it was too late and the system was ready to crash from memory exhaustion.

This doesn't mean it was a bad idea. Thing is, I believe it goes one step further and that someone making a web browser should in fact make the effort to port theirs to a virtual machine.

Although my friend Lars would blast me for saying so, applying fixed limits on memory consumption for an application is a bad idea. Even though it could solves some problems on a technical level, would lock each application into a certain space and limit expandibility of the system while also limiting the complexity of the web pages a viewer could visit.

The real solution is relocatable memory. See, although the idea of using relocatable memory isn't specific to virtual machines, they are in fact the place where it appears to have been implemented best. See, since pointers in a application running on a virtual machine aren't real pointers, but pointers to pointers to memory, it becomes possible to defragment, swap out, compress or whatever memory on the fly without the application even knowing about it. If an application frees some memory, it can easily be used by another application immediately. Relocatable heap is the best thing ever to happen to embedded systems.

So, unless you rearchitected your entire web browser to use something like auto pointers, and you made it so that you never used pointers directly to positions on the heap, but instead always used pointer relative to the beginning of the item on the heap, there's no practical way of supporting relocation from the application level. On the other hand, using a virtual machine, auto-pointers and offet based memory access is a given... well except when using things like unsafe regions in .NET.

So here's the deal, if you're going to develop for an embedded system, develop for a virtual machine on the system. It makes more sense. Using a .NET implementation with C++ would work as well, just take a hell of a long time to get through cleaning up the code to handle it. But it's worth it. The first REAL web browser to run in a VM will be the best browser on embedded devices for stability and managability.

Also, let me point out from years of experience at a browser company. Most of the time spend making a browser work on a phone is related to small device bugs. Debugging crap that behaves differently on the target then on the development machine. Using a VM would fix those problems quick and then the developers could move onto making the browsing experience better.

P.S. - Nokia, if you're reading this, dump that Safari crap and go back to Opera. Bastardizing WebKit code with Symbian code is a sin and you should burn in hell for it.

I was asked recently, an hour or so ago about what a tool chain is from a friend of mine that is currently embarking upon the world of embedded Linux development. As I explained, I came to the realization that it's a topic that could in fact use some clarification.

See, many people coming to the Linux world from another world where binary compatibility is taken for granted may see the concept of a tool chain as weird. In fact, without delving into the intense "discussions" on the topic of standized kernel ABIs by Linus could be entirely confusing since it seems illogical. So I'll write a brief explaination without getting into the decade long debates of the political workings of the Linux kernel development team.

First off, the Linux kernel is a bunch of source code that can be compiled into what feels like an infinite number of possible configurations. It is not a specific program that can be considered a constant. In the case of many operating systems, the interfaces between the kernel and the loadable drivers is relatively constant. In fact, I believe that it is still possible to load many Windows 2000 drivers on Windows Vista (if it would even be worth trying). In the case of Linux, a simple kernel upgrade will render most drivers useless without recompiling them as well.

Linux in fact is quite unique in that it may be one of only a handlful of operating systems where nearly every major component (including the kernel itself) is interchangable with 2 or more possible choices. In fact, there are circumstances where replacing the Linux kernel of an operating system with a BSD kernel without changing everything else could in fact be possible (with a lot of work).

So, what is it that actually makes up a toolchain? In the case of the Linux world, a tool chain is a group of tools, libraries, and header files that are used to compile everything for a single platform. The common components are as follows :
- binutils, in other words, the assembler, librarian, and linker
- A C compiler, GCC or Intel Optimizing Linux compilers for example
- Kernel headers, the header files from the Linux kernel of the target system (at least the config files change from target to target).
- A standard C library such as glibc or NewLib

Binutils obviously changes from target to target since the assembler must target the processor. The linker needs to target the processor, executable file type (ELF, COFF...), the ABI, and more.

The C compiler changes since the default libraries, include files, and include paths changes based on the target platform. Not to mention, the C compiler needs to know to call the right version of binutils. This is more for configuration issues than anything else. It's also important that the C complier is compiled with the same ABI as the target. Also it's useless to compile for an ARM target using a compiler targetting MIPS. Also since C++ is typically also available in the tool chain, it's important that the right compiler version is used since between GCC 2.95 and recently, there's been a great deal of movement in the calling interface used by C++. Thankfully, it appears most of this is becoming standardized now, all that's left is incompatibilities between individual interprettations of the specs.

The kernel headers define the ABI or Application Binary Interface to the kernel. This is where things like 32 vs. 64-bit system calls are describes. There are literally hundreds of options that can alter the ABI just enough to cause binary incompatibilities big enough to make executables not load. That's why the kernel header files from the specific kernel configuration for the target must be used.

The standard C library changes between targets. See, people often think that the standard C library is impossible to make compatibility issues with, but a good example would be if the header files of one C library defines a function and another defines a #define that forwards to a function of another name. So for example, an implementation of itoa() could simple do the following :
#define itoa(x) { (int) ltoa((long) x); }

So, the C library is a moving target. Also remember that the standard C library typically contains the links to the functions for dlopen(), dlfcn(), and dlclose() which are possible some of the most important library functions in all of UNIX.

Let me at this time explain what an ABI is in a short term since if it hasn't been understood until now, it needs to be. A standized ABI says that an application written for one version of a library will run against the next version of the library without recompiling first. So if the library contains functions A(), B() and C(), then next version of the library will contain them as well with the same function parameters and they'll run more or less the same way, in my opinion, good applications even keep the "functional bugs" between versions.

Linux Kernel ABI changes almost every time make config is run. Because of this, kernel modules that use the ABI need to be compiled as well for the new version. Linux is getting a little better at versioning and revision control, but things still get confusing to the user.

So, in short, every target platform has its' own toolchain. Often the only difference is the kernel headers, but it's still good to make sure you have the right C library, libstdc++, GCC generation (2.95, v3, v4), and binutils.

If you're looking for good information on making a new toolchain, I can't help much there since the few times I've done it, it was a nightmare of make config and defining tons of different prefixes.

Tags: cross-compile, linux kernel, tool chain

I've been watching the Mac PC commercials on Apple's website today. Make no mistake, they are entertaining, but I find them entertaining since apparently, Apple marketting seems to just make stuff up as they go along.

First off, let me point out that Apple is one of the best PCs I've ever owned. In fact, I just wish I could buy Macs with a license to Windows and not OS X. OS X is typically a waste of time for me. See, I work in professional video and OS X isn't very good at it. The good news is that soon Adobe will return to the platform with Premier and Avid doesn't look too bad on it.

So, let's talk about the Mac commercials, I'll only cover one or two per posting.

I'll start with "Party is Over" since it's an extremely easy one to nail.

Apple release bug fixes with a feature or two every year. They call it a release, but if we look at Apple releases of OS X, then we'll easily realize the flaw in their tech. I'll do this by adding a timeline to my OS upgrades since 2001.

In 2001, I paid $129 for Mac OS X 10.0 Cheetah. For this money, I felt that I got an operating system that was a paid beta version. Stability was so poor it was a crime and dock locked up all the time. The good part about this release is that "Force Quit" worked better than on later releases. Keep in mind that "Force Quit" is probably Apples #1 reason for less reboots since most people don't know how to do the same thing in Windows.

In 2001, I paid $129 for Mac OS X 10.1 Puma which added support for playing DVD's (I think) and provided bug fixes. For this version I had to upgrade my Mac. When I saay upgrade, I mean buy a new one since my particular iMac G3 was the only model that didn't support CPU upgrades. Funniest thing was, it was a DV Special Edition which was supposed to be the best of the iMac G3s.

In 2002, I paid $179 for Windows XP, this operating system was the first time since Microsoft started that I felt I got a good OS from them. It was pretty solid, didn't need that many reboots, most changes in the system (except computer name) could be done without rebooting finally. Also, it was the first time that DirectX actually worked well on an NT kernel. This was great since it made games on the 'pro' OS just as good as on the home user OS. What's really cool is, it runs all the code I've ever written dating back to my DOS 3.3 stuff.

In 2002, I paid $129 for Mac OS X 10.2 Jaguar. This OS was almost good enough to use every day and my Mac OS 9 friends started to consider it worth upgrading away from OS 9. So, this OS included some bells and whistles and the iApps have expanded now to included a bunch of crap I never use except iPhoto and iTunes, and iTunes I use on my PC since I have a better sound card there.

In 2003 or so, Microsoft released tons of patches and started focussing really hard on security. This was great, but sadly, it had a performance impact on Windows since the code for checking for security related issues took a lot of CPU. Good news was, at this point, Windows XP became stable enough that I began running XP for months at a time without a reboot.

In 2003, I paid $129 for Mac OS X 10.3 Panther. This was the most boring update of all the OS X versions. Basically, they charged me $129 for an OS that included a webcam chat program that wasn't compatible with anything, sold me a firewire camera that didn't implement any standards so it took me 2 months to write a Windows driver for it so I could use it with Yahoo chat. Let's not forget Expose which I still don't think I've ever found useful. Oh, and don't forget, they charged us to fix their PDF rendering bugs.

In 2004, Microsoft gave away a bunch of programs for Windows and focusses more on security and stuff. The OS was really get solid now. Drivers were no longer a problem and I'd managed at this point to go a while without upgrading the PC, but this time, I spend a few hundred bucks and got more RAM and a CPU that was 3 times as fast.

In 2004, Apple skipped their yearly upgrades, but they upgraded their iApps a few times. I didn't notice any difference since I had started using Adobe Photo Album free edition at this point. It's much more powerful than iPhoto but not as pretty.

In 2005, I paid $129 for Mac OS X 10.4 Tiger. This is what I considered to be a version 1.0 of Mac OS X and finally after paying over $500 the OS X looked good, but it was time to upgrade my Mac to get a video card that could support Quartz Extreme. The last Mac I bought for 10.1 wouldn't support it in 10.2. And since it was also an iMac, it wasn't upgradable. This time, I spent $1500 and got a PowerBook G4.

Well, in 2005/6 Apple release Mac OS X 10.4 Tiger for Intel and well, my brand new PowerBook G4 was now shit since all development shifted to x86 and I rapidly saw that the Intel version was the only one worth having, so I had to get a new notebook... again.

In 2007, Microsoft released Vista which for the most part is really damn nice. I find certain things annoying, but for the most part, they got it right and as soon as the vendors sort out the driver support, it'll be pretty solid. I just hope Toshiba releases a new ACPI bios for my 2 year old Qosmio Q20 which BTW runs Vista beautifully. No upgrades were needed in this house for Vista and we have 5 machines running it already (Including the MacBook since I'm done with OS X) BTW, I paid $199 for Vista Ultimate upgrade or whatever.

So the moral of the story kids is, no upgrades needed for Vista, but had to replace machines for Mac OS X. 5 years of service packs and patches for Windows XP for free, Apple charged me about $650 during that time. So in the same period, owning OS X was about $300 more expensive just for the software for me than Windows.

So Apple, take this and shove it. Sell your machines with Windows and ditch OS X already. It's too expensive.