O2 Script/Removing Spam pages from OWASP website (28 Jun 2010)

From

On the 28th June 2010, Paolo Perego noticed that there were a number of spam pages deteted in the OWASP website. He alerted the owasp-leaders list and we started looking at this issue. Since cleaning up these pages was more than removing one page, lets use this problem as a case study, so that we see a number of O2 scripts in action.

This script uses the O2's Media Wiki API and MediaWiki Editor scripts

Taking a screenshot of an affected page

This first script will:

• open a new instance of IE
• open one of the afected pages
• take a screenshot of the entire page
• show the screenshot in a WinForms picture box, and
• copy the screenshot into the clipboard

var ie2 = "http://www.owasp.org/index.php/How_to_convert_YouTube_to_iPhone_4_video".ie(0,0, 800,600);    
 
var screenshot = ie2.screenshot(); 
panel.clear();
panel.add_PictureBox().show(screenshot);  
panel.putBitmapOnClipboard(screenshot.bitmap());

• this is what one of the spam pages (http://www.owasp.org/index.php/How_to_convert_YouTube_to_iPhone_4_video (link doesn't work any more)) looked like
  • click on image to see full size
  • Note how it looks authentic at first sight, but at closer look they were designed to get users to download a particular software.

Once the screenshot is in the clipboard we can use the O2 MediaWiki editor tool to copy it to a MediaWiki website (in this case the o2platform.com website). This is done by opening up the page to edit and (while the image is in the clipboard pressing Paste (i.e. Ctrl+V)

Looking at the resources used in this page

the following script will show the raw xml data that can be retrieved for any page using MediaWiki's API.php

panel.clear();
var codeViewer = panel.add_SourceCodeViewer();
var wikiApi = new OwaspWikiAPI();
var targetPage = "How_to_convert_YouTube_to_iPhone_4_video";
var xml = wikiApi.parsePage_Raw(targetPage).xmlFormat();
codeViewer.open(xml.saveWithExtension(".xml"));

From that list we get the images used (which will also need to be deleted) and also a number of external links (to www.aneesoft.com)

Listing pages from a particular domain

A small variation of the previous script allows us to see all pages in the OWASP wiki that point to the www.aneesoft.com domain

panel.clear();
var codeViewer = panel.add_SourceCodeViewer();
var wikiApi = new OwaspWikiAPI();
var targetPage = "How_to_convert_YouTube_to_iPhone_4_video"; 
var xml =  wikiApi.exUrlUsage("www.aneesoft.com").xmlFormat();
codeViewer.open(xml.saveWithExtension(".xml"));

Once we have that list we will want to extract the title attribute value

var wikiApi = new OwaspWikiAPI(false);
var xml =  wikiApi.exUrlUsage("www.aneesoft.com").xmlFormat();  
var xRoot = xml.xRoot();
var pages = xRoot.element("query").element("exturlusage").elements("eu").attributes("title").values();//("eu"); 
return pages;

Finding the images used on all pages

Now that we have a list of pages we want to delete, we need to find out all unique images used by those pages

Here is a script that populates a treeview with the images that are used by all pages that contain a link to www.aneesoft.com

panel.clear();
var wikiApi = new OwaspWikiAPI(false); 
var treeView = panel.add_TreeView();
foreach(var page in wikiApi.exUrlUsage("www.aneesoft.com"))
{
	var node = treeView.add_Node(page); 
	foreach(var image in wikiApi.images(page))
		node.add_Node(image);
}

Putting it all together and deleting pages and images

This final script will:

• login with an account with admin priviledges in the OWASP Wiki
• get all pages that contain a link to www.aneesoft.com
• foreach of those pages get the list of images used
• create a unique list of images used
• delete all pages and images discovered in the previous steps

var wikiApi = new OwaspWikiAPI(false);    
var credential = @"C:\O2\_USERDATA\O2TestUsers.xml".credentials()[4];
wikiApi.login(credential.username(), credential.password()); 
if (wikiApi.loggedIn()) 
	"user logged in".info();
 
var pagesToDelete = wikiApi.exUrlUsage("www.aneesoft.com");
var imagesToDelete = new List<String>(); 
 
foreach(var page in pagesToDelete)
	imagesToDelete.add_OnlyNewItems(wikiApi.images(page));	
return pagesToDelete;
wikiApi.deletePages(pagesToDelete);
wikiApi.deletePages(imagesToDelete);
 
return "done";

• here is a sample of the logs created during the script execution:

• here is the confirmation of deletion from the OWASP's wiki 'recent changes page': http://www.owasp.org/index.php?title=Special:RecentChanges&limit=100&hidebots=0

• and as a final source code example, here is how to show the recent changes in a TreeView

panel.clear();
panel.add_TreeView()
	 .add_Nodes(new OwaspWikiAPI(false).recentChanges());
return "ok";