I was inspired by Ohmpie’s post, “Encoding an image to sound” about sound in gray scale images. One extension of his work is steganography. Steganography aims to send a message in an image without others’ knowledge that an encrypted message is attached. I started with the following image, upon which I could add data.
The following image has hidden audio data encoded in the colors used in the image.
The audio encoded in the image was extracted from this image. You can listen to it. Audio after decryption
A closer look at the image shows some of the dithering caused by the extra data.
How does it work?
For this project, I worked with X-Pixmap (XPM) files, which are trivial to manipulate in Gimp and Perl. I sorted the colors using a 3-D Hilbert curve. It had to be 3-D curve to account for red, green and blue. The Hilbert curve guarantees that not one color will differ from the next significantly. I sorted the entire spectrum into one dimension. I only used one out of every 16 colors in the palette. This leaves me with 4 bits per pixel. For this image size, this allows me to embed 34kB of data of any type. For embedding the hidden data, I sorted the colors based on the scheme described above, then dithered the color slightly to add the 4 bits. To decode, I sorted the color palette, computed the length along the Hilbert curve, modulo 16, then took the remainder, which I concatenated to re-create the data.
Posted in Uncategorized at 12:03 am by David Kellogg
Grace Hopper sat in the back of my head for years. I remember someone from long ago telling a crowd that everyone could receive a nanosecond of wire. The wire is about a foot long. Then she brought out a millisecond. That was impressive. Ms. Hopper held a resume of impossibilities. She was a college professor who joined the Navy at age 37, wrote the manual for the Mark I, became the mother of COBOL and all programming languages, became a Rear Admiral and DEC scientist. I found her again on David Letterman’s show in the early Eighties.
In the old days of physics grad school, Terry and I would hang out at TIS and talk about books. The conversation would go like this.
Terry: Did you ever buy one of those Dover books? Me: No. It’s like looking at a subject like solid state physics from the eyes of the ’60s. They never revised the thing. Terry: Well if you ever find extra change in your car seat and you want all of the equations in one place, you can blow $9.95 on the Dover version.
Dover is a publishing house that sells Pulp Physics and Pulp Linear Algebra. The paper and the printing are sub-pron, and they seem never to revise their books. We grad students never stooped to the Dover level, and would rather pay $200 for a real book.
My life came full circle as this ex-physicist who sold all of his physics and math texts suddenly found a need for linear algebra. Ironically, now that I make several times my stipend, I hunted for the cheapest linear algebra that could explain what I once knew. I came across this one,
Matrices and Transformations by Anthony Pettofrezzo. I blew all of 7 bucks on the thing. It’s my first Dover book. And it’s a good one. I’m inverting matrices like a high school genius. Eigenvectors are my gimp. Now, if only I can get back to my grad-school form.
It looks like Powerset was sold to Microsoft for $100 million. Such acquisitions remind me of all of the crazy buyouts by Yahoo. When Google spent its money on cool loss-leaders, such as Youtube, Yahoo scrambled to find basic technologies to make the company work. This is an admission that Live search is so bad that it views the poor offering by Powerset to be good technology. The worst part of this deal is that Live search is not that bad, but it shows very poor judgment to reach for straws like this.
A recent set of searches confirmed what I already believed, that the technology is not worth much. I searched for myself. I was not on the list. In fact, there is a strange inverse long-tail effect. All I saw was Inspector Gadget and sometimes Playboy video director David Kellogg. Not that I have anything against the film maker, but the average American probably *meant* the director, and that’s all you get. Goodbye long tail, hello average meaning. Then there’s usability.
Clicking on one of the down arrows gives me an infinite swirling Web 2.0 wait signal. It never stopped. Clicking on another down arrow gives me a strange window that can almost be scrolled, but only slowly. It shows you where in a theoretical page layout without pictures where the text might be found if the source page had no pictures. I know they mean to show thumbnails, but they’re not. They are semantically similar. That’s very clever.
The site is slow. Perhaps it’s the Ruby on Rails. The usual search took 4 seconds. You could do 16 Google queries by then.
I tried the query, “manamana muppet video with subtitles in swedish”. Yahoo, Live and Google all showed me the correct video as the first result. Yes, the subtitles were in Swedish. Powerset showed no results. The least this company could do is download and index filler just in case. A helpful link asked, “Did you mean monomania muppet video with subtitles in swedish?” Clicking on this link also gave zero results. There’s just no winning with these people. I thought, maybe a small company like Powserset just does not have the resources. I tried Searchme, and it also gave me the correct result. Ask had the correct result. I think what we have here is the worst web search engine anyone can currently use.
I knew it was only a matter of time before a pedantic database supporter would chime in that mapreduce is not the greatest thing since sliced bread.
Mapreduce is very cool, and the last people likely to understand this are database programmers.
What is mapreduce?
Mapreduce stems from functional programing, such as Scheme. At the University of Illinois, I thought the CS program did students a disservice by teaching them Scheme. Apparently not. The simplest map reduce function I can think of is counting words.
every good boy does good
Let’s create a map function that simply places a one (1) next to each word and sorts it.
(boy, 1)
(does, 1)
(every, 1)
(good, 1)
(good, 1)
Next let’s create a reduce function that adds the number for non-unique keys.
(boy, 1)
(does, 1)
(every, 1)
(good, 2)
That’s mapreduce. If you had 1 trillion words to count, mapreduce becomes more useful. The data can start on many nodes, then return counted and sorted to many nodes. Even better, numbers are not necessary here. URLs can be added for both keys and values, such as (”www.amazon.com”,”ebay.com”). Using this model, reverse links can be counted, monte carlo simulations can run, and the result is page rank. There are many uses. This does not include every coding problem, but it opens many doors for formerly difficult problems.
Why would database people not understand this?
Database folks appear to be in a rut. They are overly concerned with optimization. They create the index, continue to add to the index, and retrieve data very quickly using the index. What mapreduce does appears wasteful. It creates the index once, cannot add to the index, and throws away all of the work each time it is run. I think someone in the Hadoop world should solve this problem of throwing away the index, but that’s only an optimization. Optimizations do not count where a paradigm shift occurred. Now from the article.
The database community has learned the following three lessons from the 40 years that have unfolded since IBM first released IMS in 1968.
* Schemas are good.
* Separation of the schema from the application is good.
* High-level access languages are good.
Are schemas good? This implies that data should be strongly typed. Now we are getting into the strongly-typed argument that seems never to be won between Java and C vs. Perl and PHP. My guess by extension is that Perl and PHP are bad and Hadoop is bad as well.
Schemas and applications could be separated, but that makes sense only in a database world. In mapreduce, the programmer is given control over his data. It’s called freedom. This also sounds too much like MVC arguments. Yes, databases and MVC save money in some cases, but in other cases, they just hold back creativity and development.
High-level languages are good. I agree. In a way, mapreduce programs are written in a high level language in all cases. The loop that you think you write in Java or C is actually torn apart and run on many nodes. I only appears that the loop runs on one machine, yet it runs on many machines in tiny pieces.
Some of you out there should try a little mapreduce programing and see how it screws with your mind. It’s wonderful to feel different about a loop. I feel just as good doing this as when I first learned SQL.
1. Can you comment your code? No!
People ask me all the time if I can comment my code. I just won’t do it. Commenting my code simply invites lesser programmers to come along and steal my job. Remember this, commenting your code will only get you fired.
2. Can you use this/that design pattern? No!
Design patterns merely sell books and create bloated code. The answer is no. Too many devs rely on a singleton that really should have been a global. Why does no one praise global variables anymore? They are all scared by the design pattern mafia.
3. Can you stop using a global variable called temp4 and be more descriptive? No!
I’m old school. If you can’t remember all of your variables and what they do, you should not be writing code. Pure and simple, you should just sit down and remember this stuff.
4. Can you use this/that language for this project? No!
Assembly is good enough for Steve Gibson. It’s good enough for me. People who do not code in assembly regularly simply lose touch of reality quickly. Then people ask why I even need an assembler. Why not machine code? Machine code is never ever portable!
5. Can you get me some coffee? No!
People think because you are not a high-paid, suit wearing consultant, you have to do everything for them. I stop at coffee. I will get you water if I am getting a cup, but not coffee. It’s too demeaning.
6. Can you take a shower? No!
I’ve been here for 22 hours. I don’t have time to take a friggin shower.
7. Can you write down how the config node works for the rest of us? No!
See 1 above.
8. Can you fix your buffer overflow or memory leak? No!
I can find many better things to do than fix a memory leak that probably came from some else’s library.
9. Can you write this as a reusable library? No.
Why would anyone ever want to use my code?
10. Can you write your code on more than one line? No!
There is a reason for semicolons to exist. They separate code. If you can’t read my code, you should get emacs or some IDE to format it for you.
Strangely, one programmer once worked with me who said no to maybe 8 of these requests. I was flabbergasted. Here was a real-life Bartelby that refused to do everything that comes natural to most coders. I have seen multiple functions written on one line. That’s fine, but very hard to debug. I’m still not sure what to due with my temp3 and temp4 variables. Maybe they mean something. For the record I never asked another coder to get me coffee.
After Twitter’s falling apart during Mac World Expo, I though about Ted Stevens.
Ten movies streaming across that, that Internet, and what happens to your own personal Internet? I just the other day got… an Internet was sent by my staff at 10 o’clock in the morning on Friday, I got it yesterday. Why? Because it got tangled up with all these things going on the Internet commercially. They want to deliver vast amounts of information over the Internet. And again, the Internet is not something that you just dump something on. It’s not a big truck. It’s a series of tubes.
From the why-not department, I server a Facebook app from Firefox. This opens up more fluid interaction between the browser and your friends on Facebook. In this case I produced a search engine for your local bookmarks. There is no reason why you can’t just start chatting through your profile page. Here’s how I did it.
You can grab the SJS code here.
I placed this code at htdocs/fb/postmarks/index.sjs in my POW server.
Maybe more interesting than the code is how I did it. I set up a similar search engine written in PHP, starting from Facebook’s mock AJAX application. I dumped Facebook’s query string to a file. I then replayed those parameters using curl to see the response. The parameter “mockfbmltext” appeared related to AJAX text. The rest of the application was just HTML. That makes things easy. On the Facebook side, they have the guts to proxy all of my content and re-write it. All javascript is scanned. The div tags are rewritten. Proxies and dynamic rewriting is slow, but it works.
Here’s the query string part of my code.
<?sjs
pow_highlight();
var textin = ““;
if(pow_server.GET[’mockfbmltext‘]) {
textin = pow_server.GET[’mockfbmltext‘];
} else if(pow_server.GET[’mockfbmltext‘] != undefined) {
document.write(”<font color=’red‘>Please enter a search</font>“);
pow_exit();
I signed up for a Facebook API key, then filled in the blanks. These are the values I used.
Application Name: Portmarks
App Server: http://davidkellogg.com/fb/portmarks/
Canvas Page URL: portmarks
Use FBML
Website
… added on Facebook?: Yes
Who can add…: Users, All Pages
Post add URL: http://davidkellogg.com/fb/portmarks/
Post-remove URL: http://davidkellogg.com/portmarks/
Default FBML: hello
Side Nav URL: http://apps.facebook.com/portmarks
If I have a bookmarks folder called “private”, those are hidden from Facebook users. If I have a “public” folder, those are the only ones shown. Once you change the code to point to your own server (CHANGE YOURSERVER.com to your IP), it should work.
My goal was to display public and private bookmarks in my Facebook profile. It was a nice proof of concept. It would be great to see even more interaction between the browser and Facebook.
Posted in Uncategorized at 10:16 am by David Kellogg
I think Silicon Valley needs a folk song. Somehow, the crazy life here needs to be summed up by a disconnected pony-tail-sporting non-blue-shirt-wearing folk singer. In lieu of that, we have “Here Comes Another Bubble” by the Richter Scales. Thanks, TechCrunch.
