How to clean up your Twitter account

Recently I decided to get rid of all of my old tweets, specifically all of them from last year and before. I had no use for most of them and curating them would have been too much of a burden (over 66.000 tweets! so much procrastination!).

Now, there are a number of online applications to do that, but they have at least one of the following problems, fundamentally the last one:

• They pull your Twitter posts from the API, which only allows you to read at most the latest 200 tweets, so removing the older ones becomes impossible.
• Some of them get around this by asking you to upload your entire Twitter archive… which contains a lot more than just your tweets (i.e. your private messages). (EDIT: I’m being told this is no longer the case, now it just contains your public timeline)
• I don’t trust them.

So naturally I rolled my own. The code is crude but it worked for me. It uses the Twitter archive zip file as well, but since it works locally you don’t need to trust a third party with your personal data. With this I managed to delete over 60.000 tweets in a day and a half, roughly – it can’t be done much faster because of the API rate limiting, but then again, what’s the rush? 🙂

SQLAlchemy 1.05 wheels for Python 2.6 and 2.7 on Windows and Cygwin

Hi folks!

I just built Python wheels for the latest version of SQLAlchemy (1.05 at the time I wrote this). This should make it a whole lot easier to install on Windows since no compiling is required. They’re only available for the latest versions of Python (2.7.10 and 2.6.6) since it appears to be impossible to make multiple minor versions of Python coexist. I did manage to install both 32 and 64 bit versions though. On Cygwin, only Python 2.7.10 is available (I don’t think it’s possible to install Python 2.6 there, at least not using the package manager), but at least I could produce builds for 32 and 64 bits as well.

I skipped Python 2.5 entirely since pip doesn’t work there, so I assume it’s not meant to be possible. If you happen to know how to make Python wheels work on it, let me know!

I also skipped Python 3.x, but that’s just because I hate it. I don’t even know if SQLAlchemy works there and quite frankly I don’t care. 😛

Enjoy! 🙂

Download

SQLAlchemy 1.0.5 for Python 2.6 (Windows 32 bits)

SQLAlchemy 1.0.5 for Python 2.6 (Windows 64 bits)

SQLAlchemy 1.0.5 for Python 2.7 (Windows 32 bits)

SQLAlchemy 1.0.5 for Python 2.7 (Windows 64 bits)

SQLAlchemy 1.0.5 for Python 2.7 (Cygwin 32 bits)

SQLAlchemy 1.0.5 for Python 2.7 (Cygwin 64 bits)

WinAppDbg 1.5 is out!

What is WinAppDbg?

The WinAppDbg python module allows developers to quickly code instrumentation scripts in Python under a Windows environment.

It uses ctypes to wrap many Win32 API calls related to debugging, and provides an object-oriented abstraction layer to manipulate threads, libraries and processes, attach your script as a debugger, trace execution, hook API calls, handle events in your debugee and set breakpoints of different kinds (code, hardware and memory). Additionally it has no native code at all, making it easier to maintain or modify than other debuggers on Windows.

The intended audience are QA engineers and software security auditors wishing to test / fuzz Windows applications with quickly coded Python scripts. Several ready to use utilities are shipped and can be used for this purposes.

Current features also include disassembling x86/x64 native code, debugging multiple processes simultaneously and produce a detailed log of application crashes, useful for fuzzing and automated testing.

What’s new in this version?

In a nutshell…

• full 64-bit support (including function hooks!)
• added support for Windows Vista and above.
• database code migrated to SQLAlchemy, tested on:
  • MySQL
  • SQLite 3
  • Microsoft SQL Server

  should work on other servers too (let me know if it doesn’t!)

• added integration with more disassemblers:
  • BeaEngine: http://www.beaengine.org/
  • Capstone: http://capstone-engine.org/
  • Libdisassemble: http://www.immunitysec.com/resources-freesoftware.shtml
  • PyDasm: https://code.google.com/p/libdasm/
• added support for postmortem (just-in-time) debugging
• added support for deferred breakpoints
• now fully supports manipulating and debugging system services
• the interactive command-line debugger is now launchable from your scripts (thanks Zen One for the idea!)
• more UAC-friendly, only requests the privileges it needs before any action
• added functions to work with UAC and different privilege levels, so it’s now possible to run debugees with lower privileges than the debugger
• added memory search and registry search support
• added string extraction functionality
• added functions to work with DEP settings
• added a new event handler, EventSift, that can greatly simplify coding a debugger script to run multiple targets at the same time
• added new utility functions to work with colored console output
• several improvements to the Crash Logger tool
• integration with already open debugging sessions from other libraries is now possible
• improvements to the Process and GUI instrumentation functionality
• implemented more anti-antidebug tricks
• more tools and code examples, and improvements to the existing ones
• more Win32 API wrappers
• lots of miscellaneous improvements, more documentation and bugfixes as usual!

Where can I find WinAppDbg?

Project homepage:

• http://winappdbg.sourceforge.net/

Download links:

• Windows installer (32 bits)
• Windows installer (64 bits)
• Source code

Documentation:

Online

• Tutorial
• Reference

Windows Help

• Tutorial
• Reference

HTML format (offline)

• Tutorial
• Reference

PDF format (suitable for printing)

• Tutorial
• Reference

Acknowledgements

Acknowledgements go to Arthur Gerkis, Chris Dietrich, Felipe Manzano, Francisco Falcon, @Ivanlef0u, Jean Sigwald, John Hernandez, Jun Koi, Michael Hale Ligh, Nahuel Riva, Peter Van Eeckhoutte, Randall Walls, Thierry Franzetti, Thomas Caplin, and many others I’m probably forgetting, who helped find and fix bugs in the almost eternal beta of WinAppDbg 1.5! 😉

A Python example on finding connected components in a graph

Today I’ve been coding a solution for a problem we’ve encountered with @ggdaniel (cr0hn) during the development of GoLismero 2.0. It called for an implementation of an algorithm to find connected components in an undirected graph. You can find the source code at the bottom of this post.

A graph algorithm a day keeps the CS doctor away…

Suppose we have an undirected graph (connected by lines rather than arrows) in which we can find one or more “islands” of nodes that form connections to each other, but not to nodes in other “islands”. In graph theory, these “islands” are called connected components. In the image below, we see a graph with three connected components:

Now, suppose we have a set containing all nodes, and we can visit each node to know what are its neighbors, that is, the other nodes it’s connected to. We want to find all the connected components and put their nodes into separate sets. How would we do that?

An example dependency resolution algorithm in Python

I’ve been toying with dependency resolution a bit today, since it’s one of the features we plan to add to GoLismero 2.0 plugins with @ggdaniel (cr0hn). So I came up with this short example that’s reasonably fast and doesn’t use recursion at all, unlike many of the examples that I found on the net.

The basic idea is this: given a set of tasks (nodes) and the tasks that need to be performed before them, build a dependency graph and find the sets of tasks that can be run concurrently while satisfying the dependencies. For example, suppose we have tasks A, B, C and D. Task A can be run directly, it has no dependencies. Tasks B and C must be run only after A has completed, so we say B and C depend on A. Then task D depends on B and C, which in turn depend on A.

Dependency graph example

What the algorithm does, instead of traversing the graph recursively, is iteratively finding and removing from the graph all nodes that have no dependencies – that is, no arrows coming out of them. In our example, the first iteration removes node A, the second iteration removes nodes B and C, and the last iteration removes node D. And these are precisely the three batches of tasks that can run concurrently – first task A runs, on completion tasks B and C can run in parallel, and once both are finished task D can be started.

If at some point there are still nodes in the graph but we can’t find any nodes without dependencies, that means we have a circular dependency.

Circular dependency graph example

Cheating on XKCD

In case you missed it, today’s XKCD comic titled Click and Drag is simply amazing! Go check it out first, spend a few hours lost in it, and come back only when you’re done having fun. I’ll wait here. 🙂

…

Ok, you’re back. Naturally you’ll want to cheat on it at some point, to make sure you didn’t miss out on any hidden easter eggs! So let’s take a look at the web page.

The easiest route is loading the comic on Google Chrome, or Chromium. Just right click on the image and select “inspect element”. This quickly reveals how the neat trick works.

Taking a peek under the hood…

The “world” is divided into tiles of fixed size, and at all times the page loads the tile you’re currently viewing and the surrounding ones, in order to seamlessly stitch them together when scrolling. The clickable area is a map and the coordinates are used to build the URL to the images, which always follows the same pattern (north, south, and east and west coordinates). Trying out a few numbers reveals the “north” coordinate goes from 1 to 5, the “east” coordinate goes from 1 to 48 and the “west” coordinate goes from 1 to 33. Not all coordinates seem to work around the edges of the world (north 2 west 5 doesn’t work for example) and I couldn’t get south to work with manual tries. I suppose a couple empty images are used for those (one for black and one for white) but I didn’t confirm it.

The first thing I tried was just accesing the parent directory to see if directory indexing was enabled, but no such luck. Instead, I wrote this quick and dirty script in Python to download all images, using urllib to download them and shutil to write them to disk. Missing tiles are simply skipped.

This should be enough to check for easter eggs, but it’d be interesting of someone assembles a big image containing all the tiles. Let me know if you do! 🙂

Update 1: I originally missed the east coordinate, so the script was updated to try and bruteforce in all directions 1 to 10 north and south, and 1 to 50 east and west. This means a lot more HTTP requests, so I also added a pause between them as good netizens should.

Update 2: This seems to be the complete list of valid image URLs.

Update 3: A commenter pointed out somebody did assemble the entire world image! Check it out here.

Update 4: @prigazzi on Twitter pointed out this fully navegable map as well, based on Google Maps. Check it out! It’s IMHO the best one yet. 🙂

Update 5: The previous link no longer works, but this works pretty much the same way: xkcd-map.rent-a-geek.de

Updated Impacket/Pcapy installers for Python 2.5, 2.6 & 2.7

Hi folks! In a previous post I talked about using Impacket and Pcapy on Python 2.6. Since those installers are now out of date, here are fresh ones for various versions of Pcapy and Python, built against WinPcap 4.1.2. There’s also a new Impacket MSI installer that works against all Python versions.

Enjoy! 🙂

Edited 6-May-2013: updated Impacket to version 0.9.10
Edited 18-Feb-2014: updated Impacket to version 0.9.11

Download Impacket 0.9.11

impacket-0.9.11.win32.msi

impacket-0.9.11.win-amd64.msi

Download Pcapy 0.10.5

pcapy-0.10.5.win32-py2.5-winpcap4.1.2.msi

pcapy-0.10.5.win32-py2.6-winpcap4.1.2.exe

pcapy-0.10.5.win32-py2.7-winpcap4.1.2.exe

pcapy-0.10.5.win-amd64-py2.6-winpcap4.1.2.exe

pcapy-0.10.5.win-amd64-py2.7-winpcap4.1.2.exe

Hackito Ergo Sum 2012

Hi everyone. Last week I’ve attended Hackito Ergo Sum 2012, and I wanted to share with you some of the things that I found most interesting during the talks. This won’t be a detailed review of each talk, but rather an account of a few details on the talks that I personally found more interesting, in no particular order. If you’re looking for a detailed review of each talk check out this blog.

Oh, by the way. I totally made up the names of the talks. I think it’s more fun that way. 🙂

The event took place at the headquarters of the French Communist Party, and I have to say the conference room was quite impressive. It was an underground dome all covered with white metallic plates and lamps behind, giving a peculiar visual effect.

An additional advantage of this place is that some security agencies can’t send their spooks there. Hurray to the ridiculously outdated cold war laws! 🙄

One thing I didn’t like though, was that the slides were projected in a sort of tilted curved screen, making it a bit difficult to read the slides unless you were sitting in the middle. I don’t think I was the only one with this problem because I saw a lot of heads tilted sideways… 😉

Installers for BeaEnginePython, Pymsasid, PyDasm and Libdisassemble

I’ve packaged the BeaEngine Disassembler along with its Python bindings into a no-frills Windows installer. Certainly easier than manual install, and it really helps me when installing it on virtual machines. 🙂

Enjoy!

Update: Just added another disassembler package, Pymsasid.

Update: Added precompiled Windows binaries for PyDasm on Python 2.6 and 2.7.

Update: My installers were added to the Python Arsenal for RE. 🙂

Update: The Pymsasid package was updated with a small change, so the importation works just the same as loading the .py files from the current directory (it’s just an “import *” in __init__.py)

Update: Added a simple setup script for Libdisassemble 2.0. Since I had to put the sources inside a package the import statement in your scripts will have to be adjusted if using this version. With a simple try/except block you can make scripts compatible with both versions as well, if you need to.

Downloads

BeaEngine win32 installer: BeaEnginePython-3.1.0.win32.exe

BeaEngine win64 installer: BeaEnginePython-3.1.0.win-amd64.exe

BeaEngine source installer: BeaEnginePython-3.1.0.zip (run “python setup.py install”)

Libdisassemble 2.0 win32 installer: libdisassemble-2.0.win32.msi

Libdisassemble 2.0 win64 installer: libdisassemble-2.0.win-amd64.msi

Libdisassemble 2.0 source installer: libdisassemble-2.0.zip (run “python setup.py install”)

PyDasm precompiled binaries: PyDasm-1.5-precompiled.zip (run “python setup.py install”)

Pymsasid source installer: pymsasid-0.3.1.zip (run “python setup.py install”)

Heappie! + WinAppDbg

In case you missed it, Aníbal Sacco released a new tool called Heappie! to analyze heap sprays in multiple platforms. It uses PyGame and PythonCard for the GUI and Kenshoto’s VTrace as the backend. It’s really cool, check it out 🙂

Now, I never get tired of saying how great VTrace is. It’s written in Python, supports multiple platforms and quite comfortable to develop with. But alas, I tend to prefer my own debugger (call me biased if you wish!). So today I went ahead and added WinAppDbg support to Heappie!. When WinAppDbg is installed, it’s chosen automatically as the backend. If not found, it falls back to VTrace. That way we don’t lose support for other platforms, since WinAppDbg naturally only works on Windows.

This patch also adds support for 64 bit versions of Windows, in case you were thinking this was just an exercise in self indulgence. Well, it is that, but not just that. Ahem.

TL;DR

Just download the file above, go to Aníbal’s blog to learn how to use it, and exploit all the bugs! 😀

Download

Heappie-WinAppDbg.zip