| Steve: Developing on the Edge Thoughts on development, Web-services, technology and mountains. | |
21Aug Sat2010 | Y!'s Hadoop job best practises
Arun Murthy has put up
Yahoo!'s recommended Hadoop best practises.
These are good as they show what things are bad -generally
anything that bothers the namenode too much, or any work where the
input or intermediate files aren't that big. Small jobs are the
enemy. Presumably Arun's team are monitoring stat's and identifying
the troublemakers. What you could do there is just recognise these
"inefficient" jobs and schedule them differently; allowing them,
but penalising the caller.
|
| |
Posted by steve at
14:11comments
[
0
]
trackbacks
[
0
]
| 13Aug Fri2010 | How to win friends and influence people: oracle sues google
The first thing I see on the Planet Apache feed
is the
news that Oracle are suing Google over Dalvik and, implictly,
Apache Harmony
This is really going to cause fun in the open source world. I
can see a fork coming on. Can you trust Oracle not to add features
to Java just because they have patents on it?
Google Android must be wonderful and terrible for the Java team.
Wonderful, because it finally supported the Java language on mobile
phones on a platform that was actually good, finally built an
application ecosystem. This was the reason why when Sun open
sourced Java, they made sure the mobile version was GPL, so anyone
embedding it in a phone would be motivated to pay $ for the real
thing. Even if it was $0.15 a phone, at scale, that becomes profit.
Which is why Sun wouldn't give Apache the test kit for the Harmony
Runtime unless the ASF said "not to be used in mobile/embedded
devices" (as if they could), and it's hence why Java 7 development
has stalled (trouble in the Java oversight group). What development
does go on uses the term "JDK 7" to get around the stalemate. Did
all that posturing help Sun get Java into phones? Nope.
Android ships with Dalvik, which is not quite Java, according
to Stefano, and I believe him.
However, there are patent issues, and had Sun given the ASF
access to the TCK -as I believe they were obliged to do- those
would have gone away, as compliant implementations get rights to
those patents when they pass the tests. But by keeping the TCK,
Harmony couldn't pass the tests, could it?
The other funny is that if you look
at the complaint, you see at the top the names "BOIES, SCHILLER
& FLEXNER". If I am not mistaken, they are the lawyers behind
SCO's copyright lawsuit against IBM and Novell about Linux being
based on Unix. Well, that was successful wasn't it? I consider, as
I type this on my home laptop running Ubuntu 10.4 as every physical
computer I use at work also does, from the desktop to the
datacentre. It was such a good idea it created a whole web site Groklaw.
Looking forward to this. It could be entertaining.
|
| |
Posted by steve at
09:58comments
[
0
]
trackbacks
[
0
]
| 11Aug Wed2010 | Patch week
SANS is reporting that alongside this Month's MS patches, Adobe
has patches
for Flash and other things out. Not acroread, fortunately, but
for anyone who doesn't use flashblock to turn flash off, their
browser only needs to go to one subverted site for their machine to
belong to someone else
Sigh
This is one of the big problems with Virtualisation. Any VM
image you don't keep up to date is a security risk, cost of keeping
up to date is therefore proportional to the #of VM images you have.
And any backed-up VM from months earlier is danger.
SANS also have a good article on Keeping SSH keys
secure, including some find commands to locate
keys in tar files and temp dirs. Better not forget those VM disks
too though...
|
| |
Posted by steve at
09:13comments
[
0
]
trackbacks
[
0
]
| 7Aug Sat2010 | Ubuntu 10.04 upgrades ubuntu 10.04 upgrades
I am currently upgrading the last machine in the portfolio to
Ubuntu 10.04 from Ubuntu 9.04, that being the last version I liked.
The only ways to do this upgrade is a complete reinstall of the
root disk (probably the cleanest) or an upgrade to ubuntu 9.10 then
to version 10.04. If the latter is done (as I do), copy ~/.mozilla
somewhere first to stop intermediate firefox upgrades making a mess
of things.
Also: expect all thunderbird filters to get lost. This is
inconvenient if you use thunderbird as an IMAP client to a large
server which keeps many years of ASF email lists and other mail to
hand.
Recommendations
- Make a note of the filter settings
- Turn off the indexing if you have a big server full of
data
- The junk mail data gets lost too, so keep an eye on the junk
folder
- If you have >1 account, only one seems to be retained in the
upgrade. Consider noting down all the details of all accounts
Other than that, the upgrades have gone fairly well. Sometimes
during the install process it stops asking you if some config file
you've never heard of should be overwritten, and you have to look
at the changes in a file you've never seen or whose meaning you
understand. This is not a good end-user experience. And the work
desktop sound system stops working on hibernates, so it's a good
thing alsa force-restart exists to fix that.
The one thing I don't trust, yet, is the ext4 filesystem. Only
one machine -a home laptop with a new HDD- got built up with ext4
on the clean install, and after less than two months that laptop no
longer boots after the machine powered off unexpectedly.
Filesystems should not do that. I have no further data/experience
on ext4 reliability, but it does worry me. I'm curious if anyone is
using ext4 at scale in their datacentres, and if so, how reliable
its been. Of course, if your datacentre has a power system which
never fails, these problems may not show up. At least, not at
first.
|
| |
Posted by steve at
17:02comments
[
0
]
trackbacks
[
0
]
| 7Aug Sat2010 | Gelato
First, the meal
Then, time to walk
It's night, but the buildings are lit up
Everyone else is out and about too
And what is available: gelato. Ice cream.
Lovely
|
| |
Posted by steve at
14:51comments
[
0
]
trackbacks
[
0
]
| 7Aug Sat2010 | Organizational Announcement
My 7:30am doze through the clock radio was interrupted by the
announcement that our CEO is no more; he has resigned. Oddly
enough, this year's Standards of Business Conduct
video/questionnaire did include some q's on the topic of sexual
harassment.
Not one media outlet has asked me for a quote, which is probably
because they suspect I have little of interest to say on the
matter. I stick to things like Ubuntu versions, ext4 filesystem
stability and similar stuff.
|
| |
Posted by steve at
14:34comments
[
0
]
trackbacks
[
0
]
| 7Aug Sat2010 | Italy
I've
uploaded our Italy photos.
We stayed in Puglia, the bottom S.E corner of Italy; south of
Naples, East of Sicily, but without such "reputations".
The house was an old Trulli thing with these traditional roofs.
Apparently this is very quaint, but when you are staying there the
main thing is the inside is very cool when you are having your mid
afternoon siesta.
During the morning/day we'd normally be lazy, the sprog would
play in the pool and demand one more parent in there to entertain
him, which was about the hardest chore of the day.
Evenings were the time to come out, head into town, enjoy
Italian towns by night.
Where we were staying that meant a drive in, but it was so
provincial it wasn't that bad, provided you bear in mind that the
who-gives-way rules in cities are fairly ambiguous.
At night, the towns were busy with everyone walking around,
enjoying themselves, being seen. The sprog learned an important new
italian word.
Gelato
|
| |
Posted by steve at
14:29comments
[
0
]
trackbacks
[
0
]
| 29Jul Thu2010 | back from holiday
I've been offline, away in Italy. Good fun, though being utterly
offline is odd. Catching up on emails. As my colleagues have
pointed out: never go away leaving the build broken, as it
generates a lot of mail in your inbox.
|
| |
Posted by steve at
17:23comments
[
0
]
trackbacks
[
0
]
|
  | |