Map Reduce in MongoHub

Posted on 12th December 2012 in development, mongodb, Open Source Software, Software

I recently had a request to produce a list of the users who had created the most content in a system that I was building.

All of the data is stored in a MongoDB, which made this a little more challenging than a normal query. Each piece of content in the system has an embedded document with some details about the user who created it, including their username, so I immediately thought of performing a Map/Reduce on the collection to gather the data that had been asked for.

There are a number of examples on the web for how to perform Map/Reduce in MongoDB, but I will admit that I am not too familiar with the CLI for MongoDB, as I have been primarily using MongoHub to interact with the data. I knew that there was a MapReduce tab on the MongoHub interface, which I had never used before, so I wanted to try it out.

I struggled for a while to figure out exactly what needed to be entered where on the MapReduce tab and what format the entries should take, with no luck finding any examples online explaining this tab’s usage. A combination of the MongoDB docs, this very helpful thread on the mongodb-user mailing list and looking through the code of MongoHub, I finally came up with something that worked.

Map Reduce in MongoHub

Map Reduce in MongoHub

The javascript in the Map text entry is called for every document returned by the Query, which in my case is every document in my chosen collection.

The javascript in the Reduce text entry is called for every unique key emitted from the map function, with an array containing the data that accompanied each emit. In my case the key is each unique username and the data is an array containing the value 1 in each item.

By summing the contents of the array I get the number of items created by each user in the system. I later changed this to simply return the length of the ‘count’ array, which gave the same total value as each entry was 1.

The part that took me the longest to work out was the Out text entry, even though it was the simplest in the end. I was unsure as to what should be put in here and also what format the entry should take. This example of Map/Reduce in the MongoDB docs was the key to my answer.

Once I had the Output field filled in correctly I was able to run the query and create a new collection called “Ambassador” which contained the results of the Map/Reduce. I was then able to perform queries on this data to discover who had created the most content.

Hopefully this example will help someone else figure out how to use this tab in a shorter time than it took me.

comments: Comments Off

Moving to the Mac

Posted on 3rd August 2008 in OSX, Software

Background

A few months ago I was in need of a new laptop. My current Sony Vaio was starting to struggle running Visual Studio 2005/2008, IIS and MSSQL 2005 all at once.

So I started looking around for a replacement. Whilst reading reviews and generally searching for recommended development laptops, I started noticing a number of articles that were claiming that the Mac Book Pro was the fastest Windows XP laptop out there.

At the time our company was thinking of developing an iPhone version of our mobile phone software and I wanted to find out what it was like living in a non Windows OS. I have worked almost exclusively on Windows platforms since leaving Sixth Form in 1999 (wow that seems a long time ago). So I decided to give OSX a try.

When I told some of my friends they thought that I had gone mad, after all I use my laptop for work and my job is developing software. Which isn’t an issue until you find out that I build ASP.net websites and Winforms applications in C#.

Finding Applications

The first thing that I did was try and figure out the applications that I would need for everyday use of Leopard. Everyday tasks on a Windows XP machine would see me using the following applications: -

  1. Email – Outlook
  2. Calendar – Outlook
  3. Web Browser – FireFox or IE
  4. Text editing – Notepad++
  5. Instant Messaging – Windows Live Messenger
  6. Application Launching – SlickRun
  7. Note taking – Jot (Part of SlickRun) or Microsoft OneNote
  8. Playing Media – Windows Media Player, Media Player Classic or when I synced my iPod, iTunes
  9. IRC – mIRC
  10. VOIP – Skype
  11. RSS Aggregator – FeedDemon
  12. PDF Viewer – Adobe Acrobat Reader
  13. Blogging – Windows Live Writer
  14. FTP – FileZilla
  15. Screen Shots -Cropper
  16. Task Manager
  17. Remote Desktop – Microsoft Terminal Service Client
  18. Virtual Machine – Microsoft Virtual PC
  19. Document editing – Microsoft Office 2007

I spoke to some friends about what they used on the Mac, played with different applications that were out there and came up with the following list of replacements.

  1. Email – Mac Mail*
  2. Calendar – iCal* and Google Calendars
  3. Web Browser – FireFox, Safari* and I also added Camino to the list of browsers [Update:27/01/2009 - More and more I am turning to Opera as my default browser]
  4. Text Editing – TextWrangler [Update:27/01/2009 - Took the plunge and purchased TextMate and using it all of the time now]
  5. Instant Messaging – Adium (Multiple IM client support. I use it for MSN and ICQ)
  6. Application Launching – QuickSilverLaunchbar
  7. Note taking – Stickies* and Evernote YoJimbo
  8. Playing Media – iTunes* and QuickTime* + Perian
  9. IRC – Colloquy
  10. VOIP – Skype
  11. RSS Aggregator – NetNewsWire
  12. PDF Viewer – Preview*
  13. Blogging – Blogo (Also used for twitter)
  14. FTP – CyberDuck (Free), Transmit (Commercial) [Update:27/01/2009 - Using FileZilla all the time now, recent updates have made it stable enough for me to use]
  15. Screen Shots – Grab*
  16. Activity Monitor
  17. Remote Desktop – CoRD
  18. Virtual Machine – VMWare Fusion
  19. Document Editing – Neo Office

Extra applications

There other applications that I have installed which I find very useful.

  1. Growl
  2. smcFanControl
  3. SynergyKM
  4. AppZapper
  5. Delicious Library

Growl allows applications to plug into its API and use it for notifications. The notifications are then controlled by growl and the growl menu. It is possible to configure how notifications behave, how they look, how long they stay on screen and many other aspects. It also appears that notifications can be sent and received across the network, but I have not used this feature.

The second is smcFanControl because my Mac Book Pro didn’t seem to want to spin its fans very fast and tended to get very very hot. smcFanControl allows the user to set fan speeds and have profiles setup for easy switching.

It took me a while to find a working Synergy client for Leopard, however SynergyKM works excellently. I have Synergy running in server mode on a Windows Vista 64bit desktop machine, a Synergy client running on an XP laptop and finally SynergyKM running on this Mac Book Pro. This allows me to use the Keyboard and Mouse from the Vista machine, to drive all 3 machines. When my mouse hits the edge of my monitor, it jumps over to the next machine and takes the keyboard focus with it.

AppZapper is an application uninstaller that will remove all traces of the application you are attempting to remove. It always seems that an Operating System slows down after a number of months being used. This is normally because it is filling up with useless files and (in the case of Windows) registry settings. Many applications fail to uninstall themselves entirely, leaving trace elements behind in temporary files, settings and user data. AppZapper is the way to avoid this happening on your Mac.

Delicious Library lets you use the built in iSight camera to scan barcodes on your stuff. It then uses Amazon to search for the barcode and enter whatever information it can find about you things. It displays your things on shelves using images downloaded from Amazon or ones that you have assigned. Makes it quick to find out if you have something before buying it again. I managed to scan 327 DVDs and 134 books into the application in a couple of hours.

Development Applications

Normally I would install applications directly into my Windows XP installation to allow for .net software development. However on the Mac Book Pro I don’t have the option to install the framework and the development tools. Ok, so I could install Eclipse or TextMate and the mono framework and develop that way, but I need to use the full Microsoft.NET framework for my work.

I am using VMWare Fusion as a virtualization environment. This allows me to run Windows operating systems in a Virtual Machine(VM) on my Mac Book Pro, then simply take the image of the VM over to my Windows machine when I need to and run it there. I will detail my VMWare Fusion setup in another post.

Applications I found whilst researching this post

Whilst looking for links for this post I came across a number of other applications that I am going to try out. I’ll post again sometime soon with my thoughts on them.

FanControl – Like the name suggests, this is another application for controlling the fans. [Update:27/01/2009 - This is now setup as my fan control software.]
AppFresh – Keeps all of your applications up to date. Checks various update sites. [Update:27/01/2009 - I am enjoying this application, it makes keeping things up to date so simple.]
Deeper – Allows access to otherwise hidden preferences.
OnyX – A multifunction utility for Mac OS X 10.5. [Update:27/01/2009 - I use this periodically to tweak setting on my MBP]
Maintenance – A System Maintenance and Cleaning utility for Mac OS X.
FileZilla – OSX Version of FileZilla is available, can’t believe I missed this.
SuperDocker – Allows to customize parts of your Mac OS X Leopard.
LiteIcon – A simple app which allows you to change your system icons easily.
DVDHunter – A free DVD cataloguing application.
VideoLan – Media player and can be used as a streaming server.

Links

http://www.coolosxapps.net/

http://damieng.com/blog/tag/mac-freeware/
http://simpable.com/software/mac-software/

comments: Comments Off