How to clean up your Twitter account

Recently I decided to get rid of all of my old tweets, specifically all of them from last year and before. I had no use for most of them and curating them would have been too much of a burden (over 66.000 tweets! so much procrastination!).

Now, there are a number of online applications to do that, but they have at least one of the following problems, fundamentally the last one:

• They pull your Twitter posts from the API, which only allows you to read at most the latest 200 tweets, so removing the older ones becomes impossible.
• Some of them get around this by asking you to upload your entire Twitter archive… which contains a lot more than just your tweets (i.e. your private messages). (EDIT: I’m being told this is no longer the case, now it just contains your public timeline)
• I don’t trust them.

So naturally I rolled my own. The code is crude but it worked for me. It uses the Twitter archive zip file as well, but since it works locally you don’t need to trust a third party with your personal data. With this I managed to delete over 60.000 tweets in a day and a half, roughly – it can’t be done much faster because of the API rate limiting, but then again, what’s the rush? 🙂

SQLAlchemy 1.05 wheels for Python 2.6 and 2.7 on Windows and Cygwin

Hi folks!

I just built Python wheels for the latest version of SQLAlchemy (1.05 at the time I wrote this). This should make it a whole lot easier to install on Windows since no compiling is required. They’re only available for the latest versions of Python (2.7.10 and 2.6.6) since it appears to be impossible to make multiple minor versions of Python coexist. I did manage to install both 32 and 64 bit versions though. On Cygwin, only Python 2.7.10 is available (I don’t think it’s possible to install Python 2.6 there, at least not using the package manager), but at least I could produce builds for 32 and 64 bits as well.

I skipped Python 2.5 entirely since pip doesn’t work there, so I assume it’s not meant to be possible. If you happen to know how to make Python wheels work on it, let me know!

I also skipped Python 3.x, but that’s just because I hate it. I don’t even know if SQLAlchemy works there and quite frankly I don’t care. 😛

Enjoy! 🙂

Download

SQLAlchemy 1.0.5 for Python 2.6 (Windows 32 bits)

SQLAlchemy 1.0.5 for Python 2.6 (Windows 64 bits)

SQLAlchemy 1.0.5 for Python 2.7 (Windows 32 bits)

SQLAlchemy 1.0.5 for Python 2.7 (Windows 64 bits)

SQLAlchemy 1.0.5 for Python 2.7 (Cygwin 32 bits)

SQLAlchemy 1.0.5 for Python 2.7 (Cygwin 64 bits)

WinAppDbg 1.5 is out!

What is WinAppDbg?

The WinAppDbg python module allows developers to quickly code instrumentation scripts in Python under a Windows environment.

It uses ctypes to wrap many Win32 API calls related to debugging, and provides an object-oriented abstraction layer to manipulate threads, libraries and processes, attach your script as a debugger, trace execution, hook API calls, handle events in your debugee and set breakpoints of different kinds (code, hardware and memory). Additionally it has no native code at all, making it easier to maintain or modify than other debuggers on Windows.

The intended audience are QA engineers and software security auditors wishing to test / fuzz Windows applications with quickly coded Python scripts. Several ready to use utilities are shipped and can be used for this purposes.

Current features also include disassembling x86/x64 native code, debugging multiple processes simultaneously and produce a detailed log of application crashes, useful for fuzzing and automated testing.

What’s new in this version?

In a nutshell…

• full 64-bit support (including function hooks!)
• added support for Windows Vista and above.
• database code migrated to SQLAlchemy, tested on:
  • MySQL
  • SQLite 3
  • Microsoft SQL Server

  should work on other servers too (let me know if it doesn’t!)

• added integration with more disassemblers:
  • BeaEngine: http://www.beaengine.org/
  • Capstone: http://capstone-engine.org/
  • Libdisassemble: http://www.immunitysec.com/resources-freesoftware.shtml
  • PyDasm: https://code.google.com/p/libdasm/
• added support for postmortem (just-in-time) debugging
• added support for deferred breakpoints
• now fully supports manipulating and debugging system services
• the interactive command-line debugger is now launchable from your scripts (thanks Zen One for the idea!)
• more UAC-friendly, only requests the privileges it needs before any action
• added functions to work with UAC and different privilege levels, so it’s now possible to run debugees with lower privileges than the debugger
• added memory search and registry search support
• added string extraction functionality
• added functions to work with DEP settings
• added a new event handler, EventSift, that can greatly simplify coding a debugger script to run multiple targets at the same time
• added new utility functions to work with colored console output
• several improvements to the Crash Logger tool
• integration with already open debugging sessions from other libraries is now possible
• improvements to the Process and GUI instrumentation functionality
• implemented more anti-antidebug tricks
• more tools and code examples, and improvements to the existing ones
• more Win32 API wrappers
• lots of miscellaneous improvements, more documentation and bugfixes as usual!

Where can I find WinAppDbg?

Project homepage:

• http://winappdbg.sourceforge.net/

Download links:

• Windows installer (32 bits)
• Windows installer (64 bits)
• Source code

Documentation:

Online

• Tutorial
• Reference

Windows Help

• Tutorial
• Reference

HTML format (offline)

• Tutorial
• Reference

PDF format (suitable for printing)

• Tutorial
• Reference

Acknowledgements

Acknowledgements go to Arthur Gerkis, Chris Dietrich, Felipe Manzano, Francisco Falcon, @Ivanlef0u, Jean Sigwald, John Hernandez, Jun Koi, Michael Hale Ligh, Nahuel Riva, Peter Van Eeckhoutte, Randall Walls, Thierry Franzetti, Thomas Caplin, and many others I’m probably forgetting, who helped find and fix bugs in the almost eternal beta of WinAppDbg 1.5! 😉

A Python example on finding connected components in a graph

Today I’ve been coding a solution for a problem we’ve encountered with @ggdaniel (cr0hn) during the development of GoLismero 2.0. It called for an implementation of an algorithm to find connected components in an undirected graph. You can find the source code at the bottom of this post.

A graph algorithm a day keeps the CS doctor away…

Suppose we have an undirected graph (connected by lines rather than arrows) in which we can find one or more “islands” of nodes that form connections to each other, but not to nodes in other “islands”. In graph theory, these “islands” are called connected components. In the image below, we see a graph with three connected components:

Now, suppose we have a set containing all nodes, and we can visit each node to know what are its neighbors, that is, the other nodes it’s connected to. We want to find all the connected components and put their nodes into separate sets. How would we do that?

An example dependency resolution algorithm in Python

I’ve been toying with dependency resolution a bit today, since it’s one of the features we plan to add to GoLismero 2.0 plugins with @ggdaniel (cr0hn). So I came up with this short example that’s reasonably fast and doesn’t use recursion at all, unlike many of the examples that I found on the net.

The basic idea is this: given a set of tasks (nodes) and the tasks that need to be performed before them, build a dependency graph and find the sets of tasks that can be run concurrently while satisfying the dependencies. For example, suppose we have tasks A, B, C and D. Task A can be run directly, it has no dependencies. Tasks B and C must be run only after A has completed, so we say B and C depend on A. Then task D depends on B and C, which in turn depend on A.

Dependency graph example

What the algorithm does, instead of traversing the graph recursively, is iteratively finding and removing from the graph all nodes that have no dependencies – that is, no arrows coming out of them. In our example, the first iteration removes node A, the second iteration removes nodes B and C, and the last iteration removes node D. And these are precisely the three batches of tasks that can run concurrently – first task A runs, on completion tasks B and C can run in parallel, and once both are finished task D can be started.

If at some point there are still nodes in the graph but we can’t find any nodes without dependencies, that means we have a circular dependency.

Circular dependency graph example