Signed App Installation

[PythEch]

So apparently there was a python implementation of libimobiledevice here:
pymobiledevice

It is written in Python meaning that you won't even need to compile it as that is an interpreted langauge. Downside is, we need to use PyInstaller for Windows distribution because Python is not included in Windows, unlike Mac and Linux.

Anyways, if you don't want cross-language development (in other words, if you want just pure Java), we can look at that code and use it as reference to figure out how mobile installation_proxy is used. Since Python is a high-level language, it'll be easier to understand how things can be ported to Java.

I did those on a Windows machine, though it is relatively easier to get this done on Unix.

Requirements:
Python 2.7
M2Crypto
Construct

After the installation, head over to pymobiledevice and download the zip package. Extract pymobiledevice-master to C:\Python27 rename it to something like pymobiledevice otherwise Python will give syntax error. After that, fire up IDLE (Python GUI) from Start.

Now welcome to the Python Shell. You'd be surprised because all we need is 3 lines of code:

>>> from pymobiledevice import apps
>>> lockdown = apps.LockdownClient()
Connecting to device: 0aa4cc**********************************
Found pairing record for device 0aa4cc**********************************
>>> apps.mobile_install(lockdown, "C:\\Python27\\app.ipa")
Connecting to device: 0aa4cc**********************************
Connecting to device: 0aa4cc**********************************
Installing, C:\Python27\app.ipa: 5 % Complete
Installing, C:\Python27\app.ipa: 15 % Complete
Installing, C:\Python27\app.ipa: 20 % Complete
Installing, C:\Python27\app.ipa: 20 % Complete
Installing, C:\Python27\app.ipa: 30 % Complete
Installing, C:\Python27\app.ipa: 40 % Complete
Installing, C:\Python27\app.ipa: 50 % Complete
Installing, C:\Python27\app.ipa: 60 % Complete
Installing, C:\Python27\app.ipa: 70 % Complete
Installing, C:\Python27\app.ipa: 80 % Complete
Installing, C:\Python27\app.ipa: 90 % Complete
Installation Complete

Or you can just:

C:\Python27\pymobiledevice>apps.py --install=C:\Python27\app.ipa
Connecting to device: 0aa4cc**********************************
Found pairing record for device 0aa4cc**********************************
Connecting to device: 0aa4cc**********************************
Connecting to device: 0aa4cc**********************************
Installing, C:\Python27\app.ipa: 5 % Complete
Installing, C:\Python27\app.ipa: 15 % Complete
Installing, C:\Python27\app.ipa: 20 % Complete
Installing, C:\Python27\app.ipa: 20 % Complete
Installing, C:\Python27\app.ipa: 30 % Complete
Installing, C:\Python27\app.ipa: 40 % Complete
Installing, C:\Python27\app.ipa: 50 % Complete
Installing, C:\Python27\app.ipa: 60 % Complete
Installing, C:\Python27\app.ipa: 70 % Complete
Installing, C:\Python27\app.ipa: 80 % Complete
Installing, C:\Python27\app.ipa: 90 % Complete
Installation Complete

[PythEch]

So you asked

EDIT: Actually, I also need to be able to backup (but not restore) ipas through SSH. How can I do that without unsigning the ipa?

I had success with Archive command through afc. I highly doubt this is the command which is used by iTunes, iFunbox etc (I'll revEngineer later). For the development purposes, you can try this...

So when you archive an app you get signed zip in /var/mobile/Media/ApplicationArchives. I tested various apps and I can install those with iTunes, so you get indeed signed ipas. However, archiving an app without ClientOptions normally delete the app after the operation. So we need to set SkipUninstall to True. Not only that, if you don't use RemoveArchive after Archive, you'll get "Error: AlreadyArchived" when you try to Archive second time, even if you've deleted the zip.

The default pymobiledevice doesn't have Archive and RemoveArchive functions. Use my fork to get it work:
https://github.com/PythEch/pymobiledevice

Archive:

from pymobiledevice import apps
apps.archive_app(apps.LockdownClient(), "com.ifttt.ifttt")

Remove Archive:

from pymobiledevice import apps
apps.remove_archive(apps.LockdownClient(), "com.ifttt.ifttt")

Install App:

from pymobiledevice import apps
apps.mobile_install(apps.LockdownClient(), "C:\\Python27\\com.ifttt.ifttt.zip")

[alexhulbert]

This is amazing! I was almost ready to abandon this libimobiledevice thing. It should be easy enough to get this working since its written in pure Python rather than bit and pieces of C, etc. Thanks for getting this to work, its much smoother than I expected. Now I know that this project is actually possible :).

[PythEch]

Thanks for adding my name to the credits!

I'll look for better ways of backing up ipas in weekend.
P.S: I'm very happy with you being OK with Python lol

[PythEch]

I read a bit about the signing process of ipas. It turns out that we don't need anything special for backup process. IPA is a simple zip.

You can do this at home. Run iFunbox and check "Disable PNG Conversion" and "Disable bplist Conversion" from Options. For this example I'll be using "Authenticator.app". So copy ".app" folder, iTunesArtwork and finally iTunesMetadata.plist. Make directory called "Payload". Put Authenticator.app to Payload. So the filesystem should look like this:

|   iTunesArtwork
|   iTunesMetadata.plist
|   
\---Payload
    \---Authenticator.app
     ...

Then just zip everything and try to install it. It'll work.

To do that programmatically we should use "com.apple.mobile.house_arrest" service. I'll write a backup script in a few days.

[alexhulbert]

Cool! After a few minor adjustments, I've got the code so that it can run with Jython. From what I've heard, its as simple as creating a new org.python.util.PythonInterpreter, assigning it to a file, and executing python on it in the form of a string. It seems to good to be true, but I'll give it a shot and hope for the best. I've always liked python because its one of the few languages that seems to always work the way it should. I never really have those bugs where you know your code should work, but it doesn't for some reason or another. With things like C++, I always have to worry about how I'm compiling, what OS I'm on, etc. That's the beauty in cross-platform and directly interpreted languages. Anyway, I'll try to get it working and report back.

You said that we should use "com.apple.mobile.house_arrest." Why can't you just copy them from "~mobile/Applications?" Are there any advantages to using this service?

It also might be important to note that since the device will be in DFU, the actual device is going to be mounted to /mnt1 and /mnt2, while a second bare-bones ramdisk will be mounted to / using msftguy's SSH-RD. Will the services still work when executed from that alternate directory? The root partition will be in /mnt1 and the private partition (the one containing Applications) will be mounted to /mnt2. I don't believe its possible to chroot to the newly mounted directories, as I don't believe that command works properly with iOS devices.

EDIT: Would it be possible to build upon this? It should work on a regular iDevice, but I may have to change some paths around to get it to work on a DFU device.

[alexhulbert]

Just as I figured I would, I ran across a problem. Jython said it was compatible, but it turns out its not. Pymobiledevice runs on cpython, which is needed to access C. I need to use Jython, though, since I'm accessing the code through Jython's interpreter. I there any way I can get the required libraries from cpython to work under Jython's environment, or will I have to resort to something like JNI?

Here's the error I got at first:

  File "<stdin>", line 1, in <module>
  File "apps.py", line 1, in <module>
    from lockdown import LockdownClient
  File "lockdown.py", line 1, in <module>
    from plist_service import PlistService
  File "plist_service.py", line 3, in <module>
    import plistlib
ImportError: No module named plistlib

I decided to google plistlib, thinking that it was just some extra library I needed. I got it and added it to the folder. Then I got this:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "apps.py", line 1, in <module>
    from lockdown import LockdownClient
  File "lockdown.py", line 1, in <module>
    from plist_service import PlistService
  File "plist_service.py", line 4, in <module>
    import ssl
ImportError: No module named ssl

Thinking that I just had to do this for a couple libraries, I went on to download ssl.py and copy that in, too. After doing that, I got this:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "apps.py", line 1, in <module>
    from lockdown import LockdownClient
  File "lockdown.py", line 1, in <module>
    from plist_service import PlistService
  File "plist_service.py", line 4, in <module>
    import ssl
  File "ssl.py", line 359
    except socket_error as e:
                       ^
SyntaxError: mismatched input 'as' expecting COLON

I decided to look at the place I got ssl.py to see if I could figure out what I was even trying to do. I noticed they both came from websites about "CPython." Turns out that Jython and CPython are both implementations of Python. That being said, I have no clue if its even possible to use parts CPython and Jython all at the same time. Any ideas? If I'm correct, wouldn't that be like this:
C/C++ <----> CPython <----> Jython <----> Java

I'm completely clueless here, but this may shed some light on how both of these work.

[PythEch]

Cool! After a few minor adjustments, I've got the code so that it can run with Jython. From what I've heard, its as simple as creating a new org.python.util.PythonInterpreter, assigning it to a file, and executing python on it in the form of a string. It seems to good to be true, but I'll give it a shot and hope for the best. I've always liked python because its one of the few languages that seems to always work the way it should. I never really have those bugs where you know your code should work, but it doesn't for some reason or another. With things like C++, I always have to worry about how I'm compiling, what OS I'm on, etc. That's the beauty in cross-platform and directly interpreted languages. Anyway, I'll try to get it working and report back.

That's great, I.. totally forgot about Jython. It'd be interesting to see it works

You said that we should use "com.apple.mobile.house_arrest." Why can't you just copy them from "~mobile/Applications?" Are there any advantages to using this service?

AFAIK, you can't just mount /var/mobile/Applications because Apple said no. The default afc service allows you r/w to /var/mobile/Media. Not just that, I think using house_arrest is easier because you don't need to walk in to weird paths, all you need is to put AppID as a parameter, and then you are in, allowed to read (and write to some files). house_arrest is implemented in pymobiledevice.

It also might be important to note that since the device will be in DFU, the actual device is going to be mounted to /mnt1 and /mnt2, while a second bare-bones ramdisk will be mounted to / using msftguy's SSH-RD. Will the services still work when executed from that alternate directory? The root partition will be in /mnt1 and the private partition (the one containing Applications) will be mounted to /mnt2. I don't believe its possible to chroot to the newly mounted directories, as I don't believe that command works properly with iOS devices.

To be honest, I don't understand the question. I don't think the services will work (because iOS isn't booted?), but since you have /mnt2 (/var/mobile) you don't even need to use Apple's limited services. You are root, you have all the control, so just read every folders in /var/mobile/Applications and package them nicely. Then, we also have backups of applications because we have now Documents and Library directories (though some applications store their information on Keychain).

EDIT: Would it be possible to build upon this? It should work on a regular iDevice, but I may have to change some paths around to get it to work on a DFU device.

I don't think you can use Bash scripts on non-jailbroken iDevice but, the only advantage of using that is, you don't need to write code (you have the code ready).

Just as I figured I would, I ran across a problem. Jython said it was compatible, but it turns out its not. Pymobiledevice runs on cpython, which is needed to access C. I need to use Jython, though, since I'm accessing the code through Jython's interpreter. I there any way I can get the required libraries from cpython to work under Jython's environment, or will I have to resort to something like JNI?

Like I said I totally forgot about Java implementations. That is a very good idea, I'll try to get those work. Though I can't say anything before I get my hands on.

Edit:

So the problem is, SSL module was introduced into CPython 2.6. You have installed Jython 2.5 (because it is the stable one), and SSL module uses Python 2.6 only features (except syntax was changed in 2.6).

So I tried that in Jython 2.7 beta 1, this time I have this error:

Jython 2.7b1 (default:ac42d59644e9, Feb 9 2013, 15:24:52)
[Java HotSpot(TM) 64-Bit Server VM (Oracle Corporation)] on java1.7.0_40
Type "help", "copyright", "credits" or "license" for more information.
>>> from pymobiledevice import apps
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "pymobiledevice\apps.py", line 1, in <module>
    from lockdown import LockdownClient
  File "pymobiledevice\lockdown.py", line 1, in <module>
    from plist_service import PlistService
  File "pymobiledevice\plist_service.py", line 1, in <module>
    from usbmux import usbmux
  File "pymobiledevice\usbmux\usbmux.py", line 21, in <module>
    import socket, struct, select, sys
  File "C:\jython2.7b1\Lib\select.py", line 14, in <module>
    import Queue
  File "C:\jython2.7b1\Lib\Queue.py", line 8, in <module>
    from collections import deque
  File "C:\jython2.7b1\Lib\collections.py", line 12, in <module>
    import heapq as _heapq
  File "C:\jython2.7b1\Lib\encodings\__init__.py", line 85, in search_function
    norm_encoding = normalize_encoding(encoding)
  File "C:\jython2.7b1\Lib\encodings\__init__.py", line 69, in normalize_encodin
g
    return '_'.join(encoding.translate(_norm_encoding_map).split())
TypeError: translate() only works for 8-bit character strings

There are some weird things going on. Maybe it's too good to be true like you said. :( While, I agree that having a direct access to the pymobiledevice would be a great feature but, in worst case scenario we can make a python script to install, backup apps and upload/download files with parameters. e.g:

script.py --install=<path>
script.py --backup=<appid>
script.py --upload=<path>
script.py --download=<path>

[PythEch]

Signed backup script, run this on your computer:

https://ghostbin.com/paste/jao79

As for Jython problem, I solved those library problems but now I am asked to install M2Crypto and Construct libraries. I can't just install them straightforward:

building 'M2Crypto.__m2crypto' extension
error: I don't know how to find (much less run) SWIG on platform 'java'

So I searched some stackoverflow questions, it seems that http://jepp.sourceforge.net/ can be a better alternative for this problem.

[alexhulbert]

This should clear up all the confusion on DFU, etc.:

On all the older processors (Not A5), there's a bootrom exploit that's packaged with limera1n (I forgot what it's called) that can be used to upload a custom ramdisk to any A4 device while its in DFU mode. Variations of this exploit also make up most of the A4 tethered/semi-tethered jailbreaks you see (ex: opensn0w). Using a tool like this, you can upload a custom ramdisk containing some shell scripts, etc. and modify the root filesystem before the device has eve booted! The only problem is that this exploit only works for devices with A4 processors (like iPT 4G). I keep hearing rumors that evasi0n has one for A5+ in their alleged exploit arsenal, but I'm a little skeptical. Either way, its a nice tool to have, since manually backing everything up under DFU mode can take forever. As for the shell script you included, that won't go to waste :). I'm going to modify that so it can be used to backup data rather than apps.

Right now I'm looking at ca.py (the one that references M2Crypto) and it looks like its not doing anything special. From what I can see, it might be best to reimplement the cryptography part using something a little more cross-platform friendly. All the functions that are being used (and there's only 1 or 2 dozen) clearly state what their doing. For example, this code seems simple enough to port:

def generateRSAKey():
    return RSA.gen_key(2048, m2.RSA_F4)

def makePKey(key):
    pkey = EVP.PKey()
    pkey.assign_rsa(key)
    return pkey

The only two functions I'm worried about are makeRequest and makeCert, but those are heavily commented.

As for construct, I think that will work just fine with jep. It installs the same on different OSs, so that won't be a problem.

Anyway, I'm starting to port ca.py now. I'll let you know what happens.

[PythEch]

Hm, I was writing relatively a bit better version of that:
https://ghostbin.com/paste/z9qjo

if backup_app_data:
        if "Library" in files:
            recursive_save(join(path, "Container", "Library"), join('.', "Library").replace("\\","/"))
        if "Documents" in files:
            recursive_save(join(path, "Container", "Documents"), join('.', "Documents").replace("\\","/"))

This should do everything you want. Keep in mind that some apps store their info on Keychain. Keychain can be extracted from iTunes backup. Mobilebackup is already included in pymobiledevice. If the device is jailbroken just download keychain-2.db from the device.

For Jython, what I did was installing both 2.5.2 and 2.7b1 versions. After that copy and paste ssl.py and plistlib.py from jython2.7b1/Lib/ to jython2.5.2/Lib.

Good luck with the port process :)

P.S: Also thanks for explaining everything detailed, it's fun to work on something together with a nice person like you

[alexhulbert]

I have one question before I start working on this thing: Will I be able to somehow include M2Crypto and Construct in Jepp natively? Or will it be different per OS?

[PythEch]

Well, I have the same question in my mind. I'm on a trip so I can't just boot Linux here, when I come back to home I can test it for you, but probably you'll have more time to test that.

[alexhulbert]

Okay! It seems pretty straightforward, so I'll try it out and post my results here later.

[PythEch]

Good luck, here is the improved version of the backup script (FYI, you've said bash script but this is a python script :D):

https://ghostbin.com/paste/j98jj

[alexhulbert]

Aha! Look at this!

[alexhulbert]

Ironically, it seems that now Jython might be better suited. I was reading about how to import packages using Jepp, and somebody said to use Jython. I read the exact opposite, though about 5 minutes earlier.

[PythEch]

If Jython worked, it'd be awesome because from what I've seen it generates .class files which can be used by any Java program easily. The problem is I can't get Jython to work... We might need someone else to fix this part of the project.

[alexhulbert]

I thought that too. Turns out that part of the project is depricated. All you have to do now is just use a function to execute a method. I tink its exec or something. Check this out!

[alexhulbert]

All I think you need to do is extract the packages to a temporary folder, and put from pkgutil import extend_path __path__ = extend_path(<tmppath>, "construct") at the top of every file that uses construct, where tmppath is the folder containing the construct package. M2Crypto should work too

[alexhulbert]

Aha! Another breakthrough! This is exactly what we're looking for! Its specific to packages located in the site-packages folder!

[PythEch]

I don't understand what you are doing (sorry :( ), since M2Crypto and Construct are platform-dependant (although cross-platform), in either way we'll need python to be executed. If the solution you have provided will need compiled libraries of M2Crypto and Construct, it is wasted effort because the same can be achieved by executing python script like I said before.

I think we need to find java equivalent of those libraries to succeed.

Edit:
Lol, I've found this on wikipedia:

A port to Java is available on GitHub. Examples in Java, the ethernet header (layer 2)

That's great because you've said pymobiledevice doesn't do anything special with M2Crypto. Construct is no problem. If you can port ca.py, I'm very positive that we can burry this problem like forgotten stories of history.

[PythEch]

It might be a better idea to scrap ca.py and write java equivalent:

http://www.mayrhofer.eu.org/create-x509-certs-in-java

I believe this is the original ca.py file:

https://github.com/Hypernode/M2Crypto/blob/master/demo/x509/ca.py

I don't know why this is needed, there may be a workaround. When pymobiledevice paired my device for the first time, I got this alert:

http://www.macobserver.com/imgs/tmo_articles/ios7trustconfirm.jpg

When I delete .pymobiledevice folder (which is stored in %homepath%), thus deleting pairs, it tried to pair with the device again. Why I am saying this is, when using iFunbox, I don't get this alert on my device.

[alexhulbert]

Hi, I'm back! I just saw your message. I whipped this up in about 15 min., so its bound to have some bugs, but here's a prototype for the first function in ca.py
The only thing I'm worried about is what I'm calling a string and what I'm calling a byte. I don't know how python handles the two, but it seems to be somewhat different than java.

public static String convertPKCS1toPKCS8pubKey(String raw_data) {
    byte[] subjectpublickeyrsa_start = Utils.fromHex("30819E300D06092A864886F70D010101050003818C00");
    raw_data = data.replace("-----BEGIN RSA PUBLIC KEY-----\n","");
    raw_data = data.replace("-----END RSA PUBLIC KEY-----","");
    byte[] data = base64.decode(data);
    byte[] starts = Utils.fromHex("30818902818100");
    if (data.startsWith(starts)) {
        byte[] newstarts = Utils.fromHex("308188028180");
        for (int i = 0; i < starts.length; i++) {
            data[i] = newstarts[i];
        }
    }
    byte[] newdata_raw = new byte[](subjectpublickeyrsa_start.length + data.length);
    for (i = 0; i < subjectpublickeyrsa_start.length + data.length; i++) {
        if (i < subjectpublickeyrsa_start.length) {
            newdata_raw[i] = subjectpublickeyrsa_start[i];
        } else {
            newdata_raw[i] = data[i - subjectpublickeyrsa_start.length];
        }
    }
    byte[] newdata = base64.encode(newdata_raw);
    String res = "-----BEGIN PUBLIC KEY-----\n";
    for (int i = 0; i < (newdata.length/64 + 1); i++) {
        for (int j = i*64; j <= (i+1*64); j++) {
            res += new String(newdata.toCharArray()[j]);
        }
        res += "\n";
    }
    res += "-----END PUBLIC KEY-----";
    return res;
}

EDIT: Come to think of it, do we even need to port functions that don't rely on M2Crypto? facepalm

[alexhulbert]

Also, I put the question out there just in case there's a way do use packages natively in Jython. Here's the question: http://stackoverflow.com/questions/20805980/reference-m2crypto-and-construct-from-within-jython

[PythEch]

Aha, I found this in lockdown.py:

        record = readHomeFile(HOMEFOLDER, "%s.plist" % self.identifier)
        if record:
            pair_record = plistlib.readPlistFromString(record)
            certPem = pair_record["HostCertificate"].data
            privateKeyPem = pair_record["HostPrivateKey"].data
            print "Found pairing record for device %s" % self.udid
        else:
            print "No pairing record found for device %s" % self.identifier
            return
        if False:
            if sys.platform == "win32":
                folder = os.environ["ALLUSERSPROFILE"] + "/Apple/Lockdown/"
            elif sys.platform == "darwin":
                folder = "/var/db/lockdown/"
            pair_record = plistlib.readPlist(folder + "%s.plist" % self.identifier)
            print "Using iTunes pair record"

Apparently it can use iTunes pair records but developer forgot to remove "return". I've checked that folder and yes, they do exist. We can use iTunes pair records and scrap ca.py, but this will break Linux support (unless Apple starts to think having iTunes on Linux is a good idea). But that doesn't matter, openssl is installed on most Linux distros so we can ShellExecute a command to generate CA certificates. I think I can handle this side of the project when I come back to home. Right now I'm editing the file to use iTunes pair records instead of generating its own.

So what I thought is true, iFunbox and several other tools use iTunes records.

[alexhulbert]

That's awesome! I can't believe it fit together so perfectly! I'll work on my side of the project in the mean time.

[PythEch]

My fork is online again with before said changes:

https://github.com/PythEch/pymobiledevice

Edit:
Just confirmed that iFunbox is using iTunes' pair:
http://i.imgur.com/fzYurrp.png

Also it's funny to see that iTunes will give error 0xE8000003 forever when you delete folder "%AllUsersProfile%\Apple\Lockdown". Apple forgot to use mkdir? haha

[alexhulbert]

I tried to sent a comment about an hour ago, but I guess it didn't work. Anyway, everything works except for sockets. I get this error:

  File "<stdin>", line 1, in <module>
  File "pymobiledevice\lockdown.py", line 21, in __init__
    self.c = PlistService(62078,udid)
  File "pymobiledevice\plist_service.py", line 12, in __init__
    self.connect(udid)
  File "pymobiledevice\plist_service.py", line 15, in connect
    mux = usbmux.USBMux()
  File "pymobiledevice\usbmux\usbmux.py", line 222, in __init__
    self.listener.listen()
  File "pymobiledevice\usbmux\usbmux.py", line 222, in __init__
    self.listener.listen()
  File "pymobiledevice\usbmux\usbmux.py", line 191, in listen
    ret = self._exchange(self.proto.TYPE_LISTEN)
  File "pymobiledevice\usbmux\usbmux.py", line 185, in _exchange
    recvtag, data = self._getreply()
  File "pymobiledevice\usbmux\usbmux.py", line 164, in _getreply
    resp, tag, data = self.proto.getpacket()
  File "pymobiledevice\usbmux\usbmux.py", line 106, in getpacket
    dlen = self.socket.recv(4)
  File "pymobiledevice\usbmux\usbmux.py", line 51, in recv
    raise MuxError("socket connection broken")
pymobiledevice.usbmux.usbmux.MuxError: socket connection broken

I'll send you the patched python files tomorrow. You have to modify some stuff to work with jython

[PythEch]

That's weird, I didn't encounter that error. I guess it was temporary and the changes I made has nothing to do with usbmux. I'm waiting for your patch.

Edit:

Are you talking about jython? If so yes, I have the same problem:

Jython 2.5.2 (Release_2_5_2:7206, Mar 2 2011, 23:12:06)
[Java HotSpot(TM) 64-Bit Server VM (Oracle Corporation)] on java1.7.0_40
Type "help", "copyright", "credits" or "license" for more information.
>>> from pymobiledevice import apps
>>> apps.LockdownClient()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "pymobiledevice\lockdown.py", line 21, in __init__
    self.c = PlistService(62078,udid)
  File "pymobiledevice\plist_service.py", line 12, in __init__
    self.connect(udid)
  File "pymobiledevice\plist_service.py", line 15, in connect
    mux = usbmux.USBMux()
  File "pymobiledevice\usbmux\usbmux.py", line 220, in __init__
    self.listener = MuxConnection(socketpath, BinaryProtocol)
  File "pymobiledevice\usbmux\usbmux.py", line 157, in __init__
    self.socket = SafeStreamSocket(address, family)
  File "pymobiledevice\usbmux\usbmux.py", line 37, in __init__
    self.sock = socket.socket(family, socket.SOCK_STREAM)
  File "C:\jython2.5.2\Lib\socket.py", line 1200, in __init__
    _sock = _realsocket(family, type, proto)
  File "C:\jython2.5.2\Lib\socket.py", line 586, in _realsocket
    assert family in (AF_INET, AF_INET6), "Only AF_INET and AF_INET6 sockets are
 currently supported on jython"
AssertionError: Only AF_INET and AF_INET6 sockets are currently supported on jyt
hon

and now this:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "pymobiledevice\lockdown.py", line 21, in __init__
    self.c = PlistService(62078,udid)
  File "pymobiledevice\plist_service.py", line 12, in __init__
    self.connect(udid)
  File "pymobiledevice\plist_service.py", line 15, in connect
    mux = usbmux.USBMux()
  File "pymobiledevice\usbmux\usbmux.py", line 218, in __init__
    self.listener.listen()
  File "pymobiledevice\usbmux\usbmux.py", line 218, in __init__
    self.listener.listen()
  File "pymobiledevice\usbmux\usbmux.py", line 187, in listen
    ret = self._exchange(self.proto.TYPE_LISTEN)
  File "pymobiledevice\usbmux\usbmux.py", line 181, in _exchange
    recvtag, data = self._getreply()
  File "pymobiledevice\usbmux\usbmux.py", line 160, in _getreply
    resp, tag, data = self.proto.getpacket()
  File "pymobiledevice\usbmux\usbmux.py", line 106, in getpacket
    dlen = self.socket.recv(4)
  File "pymobiledevice\usbmux\usbmux.py", line 51, in recv
    raise MuxError("socket connection broken")
pymobiledevice.usbmux.usbmux.MuxError: socket connection broken

[PythEch]

I think Jython gives us too much headaches. Let's keep it simple and distribute the python part with PyInstaller. Even Tinyumbrella has two seperate versions which is written in pure Java.

From PyInstaller homepage:

PyInstaller is a program that converts (packages) Python programs into stand-alone executables, under Windows, Linux, Mac OS X, Solaris and AIX.
...
The main goal of PyInstaller is to be compatible with 3rd-party packages out-of-the-box. This means that, with PyInstaller, all the required tricks to make external packages work are already integrated within PyInstaller itself so that there is no user intervention required. You'll never be required to look for tricks in wikis and apply custom modification to your files or your setup scripts. As an example, libraries like PyQt, Django or matplotlib are fully supported, without having to handle plugins or external data files manually.

Using

pyinstaller --onefile script.py

just works. If it doesn't work try

C:\Python27\Scripts\pyinstaller.exe --onefile script.py

[alexhulbert]

Sorry, I accidentally clicked "close" instead of "comment"

[alexhulbert]

I finally have access to my laptop. I'm doing everything now!

[alexhulbert]

Currently, what certificates work? Just python ones? If libimobiledevice certificates work, we can compile the C code for Linux/Mac, and Windows and run that from pymobiledevice rather than actually porting ca.py. I would say that we could compile ca.py directly to an exe, but it requires M2Crypto. Hmmmm....

[PythEch]

Both libimobiledevice and iTunes certificates don't have issuer names, that's the reason why they don't work. It seems, Java is hardcoded to not allow empty issuer. Probably it can be monkey-patched using reflection, but I don't think it worths the trouble, that's shady and some unnecessary work. Perhaps I may try to do it when every problem is fixed, just for the sake of using iTunes certificates.

Anyway, yes, just python ones. But don't worry, it's possible to create certificates using Java. I think nycs can do it, right now I'm trying undocumented classes. In worst case scenario, we'll need to use BouncyCastle, don't think it'll be a problem.

Converting (not compiling) ca.py to an exe is possible with PyInstaller (better than py2exe and its competitors) with M2Crypto (actually it handles that part automatically), remember, I recommended using that tool to make a command line utility. But practically a lot of things can go wrong. Like catching exceptions becomes a lot harder.

If we really need a binary, we can use openssl.exe. It's a real native binary, stable, cross-platform and we don't need to compile anything.

---

I guess we can seperate the tasks. I think I can handle the ca.py part. But junix-sockets seems like suits better to you.

[alexhulbert]

Great! I UNIX sockets seems fun. The repo I'll be using is here. It seems pretty easy. I'm going to skim through the code and see if I can figure out how everything works.

[alexhulbert]

EDIT: I googled jnr-unixsocket to find some documentation when I stumbled upon junix-sockets. I remembered you talking about that in your previous comment, so I investigated it. You're right, junix-socket seems like a much better library.

[PythEch]

Lol, I googled to remember the library's name, junixsocket was 5th result so I assumed it was the same library that you were talking about. If you need any help, I'd be glad to do it

EDIT:
ca.py is almost complete, I'll probably push a commit a few hours later when I get back to home.

[alexhulbert]

UNIX sockets are going pretty well too! There's just one problem. We need to reimplement the select function. I googled select, IBM has a pretty good description:

The select() function allows the process to wait for an event to occur and to wake up the process when the event occurs. In this example, the select() function returns a number that represents the socket descriptors that are ready to be processed

The last part is probably different, since python's select returns three values.

---

I know everything's probably coded wrong. As soon as I have access to my laptop, I'll check everything out.

[PythEch]

I know I said I'll push in a few hours but I was stuck at loading public key. I finally found out that we need to pass it as DER format.

Generating CA certificate was pretty straightforward. That's why I said it's almost done. I'll try to help you about select when I finish the last step in certificate generation.

Oh and, other than that, it works

EDIT:
Everything is fixed

[PythEch]

Is there anything missing other than UNIX sockets? I haven't found a solution for that yet except a stackoverflow question that recommends using jnr-unixsocket instead of junixsocket. I'll try that later.
Both pymobiledevice and jmobiledevice are looking good.

I can also help developing jmobiledevice if you need it.

[alexhulbert]

I can also help developing jmobiledevice if you need it.

I'm good for now. You've already done certificate.py, so I'll handle this part of the port. Could you work on the UNIX socket part? I just can't visualize how the code is going to work. Plus, I'm much better at Java then Python.

---

You have probably seen the flurry of activity with JMobileDevice. I was on a very long plane ride, so I wanted to take advantage of their amazingly fast internet. Its so ironic that some of the fasted internet I've ever used was on a plane :/. I got a little carried away with the programming, so I'll explain what's going on.

After seeing libimobiledevice-wrapper and ios-driver, I decided that our port would be better off in another repo. Those repositories are a complete mess, with some changes in one repo and some changes in another. I don't want to make that mistake.

I decided to split the /src/com/alexhulbert/jmobiledevice folder into its own repo using the method described in this article. After that, I modified the .git folder so that the new repo pointed to Triforce1/JMobileDevice.

Then I realized I was stuck with a problem: The .java files were in the root of the repo; you would have to add the dependancies, python code, etc. yourself in order to compile it. That's when I got the idea to make two branches: a code branch and a master branch.

The code branch is just the jmobiledevice java files. No folders or anything. The master branch is just an empty netbeans project with all of the dependencies.

I made a submodule (link) to the code branch inside of the src directory so that the master branch contains the code branch within it. Then I linked to the code branch inside of Triforce1/Icew1nd so that when you update the code branch, both JMobileDevice and Icew1nd automatically get updated.

Think of it like javascript callbacks, but for git. As you can imagine, that's not the easiest thing in the world. I haven't felt this confused since I last opened xorg.conf :).

---

Anyway, the migration is done now and all of the code should automatically synchronize throughout all 3 repos (JMobileDevice, pymobiledevice, and Icew1nd). Currently, you can execute a plethora of Shell commands like cd, mkdir, and mv. I plan to add some more (like touch).

I had to do a couple of small fixes in the Python, but everything should work in both regular Jython and in embedded Jython.

Once JMobileDevice is done, I'd like to add some functionality from the actual libimobiledevice. This may sound hard, but its actually pretty strait-forward. You just make (or attach to) a lockdown instance and use simple send and receive commands. There's not that much logic involved. Finishing Icew1nd should be pretty easy too. I just need to finish reverse engineering iCloud.

[PythEch]

Plus, I'm much better at Java then Python.

Yeah, what you say makes sense. I have to admit, that's vice versa for me.

I got a little carried away with the programming, so I'll explain what's going on.

It was hard to understand what happened, thanks for explaining that before I ask.

Maybe not jmobiledevice but, I can help about Icew1nd like beta testing etc. I remember that you needed an A4 device

[alexhulbert]

Here in America, we're having some internet security problems too :|. Evidently the NSA's been spying on us...

---

For the A4 device, the process is going to be much more complicated than I though. I think a friend of mine has one he's not using. I'll try to persuade him to let me borrow it. The process I'm going to do is at issue #3.

---

I'll post some more info on the current status of the project later today.

[alexhulbert]

Expect a huge push within a couple days! I'm working on diagnostics_relay and I've got a great system down. It involves BigIntegers and a lot of bitwise/binary operators :).

[PythEch]

Great job! Sorry for not being active, I'm going to be very busy due to school for weeks :/

I've looked into UNIX Sockets part, I think I've found what should I do, but I didn't have much time later. Wish Jython had this feature just like JRuby

At least we made it working on Windows, so the development can continue.

[PythEch]

Aha, this might seem small but I finally got it working to a point where usbmux.py successfully recognizes my device's UDID. Right now the code is a little buggy, I'll push a commit some time.

EDIT:
And now, it gives PasswordProtected error and successfully asks for Trust This Device.

Other than that, when ~/.pymobiledevice records persist, it stops reading at a time.

[alexhulbert]

Awesome! I've been doing a huge chunk of Icew1nd in my study halls :P. (Like right now!) I'm working on the iCloud portion and the GUI right now. I wonder how they'll turn out!

[alexhulbert]

Lol. I saw you starred mountainstorm/MobileDevice. I was looking at it thinking it looked hard to port, but its actually really easy! I bet I could port some of the files. They're just sending out and receiving plists! If I have any free time, I'll port some stuff from MobileDevice over.

Currently, I'm working on building that chat program I was talking about. IRC is really stupid if you ask me...

Also, I'm going to upload the GUI (finally). I just need to tidy up some code and include a font.

[PythEch]

Sorry to say this but, actually, it's just a wrapper for MobileDevice.dll / MobileDevice.dylib, so no love for Linux.

And what's the point of porting it, pymobiledevice is already finished? :D

I starred that because I liked it as an API, I'm "beautifying" some ugly mess of pymobiledevice right now. Adding docstrings, removing some of its magic, fixing inconsistent naming convention etc.

Also seems like you didn't get notifications of my replies, probably because I deleted that branch. I don't want to repost so, I'll just leave the link here:
PythEch/pymobiledevice@74bd251

I'm looking forward to the GUI release :)

[alexhulbert]

Sorry to say this but, actually, it's just a wrapper for MobileDevice.dll / MobileDevice.dylib, so no love for Linux.

Actually, just files like plistService.py are a wrapper. In files like afc.py and springboard.py, functions made up mostly of sendPlist and recvPlist commands. Those files can be ported easily. I can do them sometime in the future if need be.

And what's the point of porting it, pymobiledevice is already finished? :D

Interfacing more of iOS's protocols to Java could help make bigger programs, such as iTunes for linux possible. It opens up a huge variety of things you can build and create.

I starred that because I liked it as an API, I'm "beautifying" some ugly mess of pymobiledevice right now. Adding docstrings, removing some of its magic, fixing inconsistent naming convention etc.

Sweet! Tidy code is always so much better in the long run. Feel free to change around any methods and I'll modify the Java wrapper accordingly.
.

Also seems like you didn't get notifications of my replies, probably because I deleted that branch. I don't want to repost so, I'll just leave the link here:
PythEch/pymobiledevice@74bd251

I literally have no idea what to call it. It would take a lot of major refactoring to name it something else, so we should only change the name when one of us thinks of something cool. Your right, Jymobiledevice sounds weird.

I'm looking forward to the GUI release :)

Awesome use of the shipit squirrel :). Anyway, I've finished it, but there's another problem with my laptop. The hard drive is still perfectly fine (thank god), so I'm going to back that up to my local NAS box (its 500+ GB of data to backup). Once that's done backing up, I can recover the files from it and push them to the main repo. Square trade said that something's most likely defective in my laptop and they're just give me the money back for it. I've still got my old fallback computer, though. So I will continue to develop when I can.

[PythEch]

Haha, even though we had the same idea, I failed to understand what you meant, oops.

Look at this:
https://ghostbin.com/paste/6b599

Not only that it's now easier to read/write files, we can also seek, tell and read line by line (__iter__) without reading the whole file. And it also supports with statement (though this shouldn't really matter in Java).

I especially endeavored to make it look like open() in Python.

e.g

with afc.AFCFile('test.log', 'r') as f:
    for line in f:
        print line

Actually, Cython bindings (yet another Python implementation, duh!) of libimobiledevice can help a lot too.

I didn't had a good idea for name, but I was preparing the license, so I thought it was the right time to ask it. Anyway, I don't think anyone's gonna use pymobiledevice (jython) directly, instead of JMobileDevice.

As for your computer problems... I have a laptop from 2008 with 2 GiB RAM running on Windows 7 in my home, and it's still functioning properly with frequent use. We haven't had any HDD problems yet, something's strange with yours :/

Fortunately it's good to hear you have backups, something I occasionally do.

Also, if you have an SDD in your new computer, that may be the problem. A friend of mine had a lot of problems with OCZ, and there was a bug in firmware of my Corsair (firmware upgrade solved the problem).

[alexhulbert]

Awesome! That code looks so much better than the original. Anyway... I've got a surprise for you ;). Its an atom invite! I finally got mine (via personal request) and I'd like to send you one of my personal invites to you as a thank you for all your hard work on this project. Is there any email in particular you'd like me to send it to? Also, you said you had a Cyder clone. Is that finished? I've been looking for one for ages.

EDIT: I've sent in my laptop and I'm going to get one that's hackintosh-compatible. I'm finally going to learn Objective-C :P

[PythEch]

That's very kind of you :) Well, I think, I should accept then: [email protected]

Thank you very much!

---

That Cyder clone is written in C# which was internally named "WinPie" (lol). I was working on it before I met you. It was functioning properly enough for a beta; you could add repos, search for packages, view depictions (the webpage you see before installing a package) even though IE failed so hard, and finally, download debs. I don't plan making it able to install debs, there are lots of tools for that job. We need a tool to download packages, not installing them. No more weird errors for missing iTunes libraries.

I'll upload pics and the source code/binary in a few hours later when I get back to home. It is still very buggy though, and somewhat slow. Probably I'll need to switch to SQLite from LINQ.

The way it works surprised me, it was possible to emulate everything using simple HTTP requests. Oh and 7zip to decompress bz2 files :D (dot net is too slow)

And the best thing is, you can download paid packages using your Cydia ID!

---

I have never been successful with hackintosh, is there anything you can recommend so I can buy that next year? I asked this so many times but never got a good answer.

[PythEch]

I'm back! The internet was broken for 2 days...

I don't think you'd want to use this half-baked thing but nevertheless I released that.

Some pics:
http://imgur.com/MzyA1OD,iRuo3X5

I'll try to finish it in summer

[alexhulbert]

Well, I "finished" the chat server! It's sloppier than your Cyder Clone, though. It preforms XHR request twice per second, so its slow as hell. You'll have to let me know to come on in advance, too. I haven't set up logging and it doesn't email me messages I don't hear yet. But I'll continue to work on it. You can see it at chat.alexhulbert.com. The GUI for Icew1nd looks nice, too! Here's a screenshot:

---

The problem with Hackintoshes is that each computer is different. You can see some guidelines and a compatibility list at tonymacx86.com.

[PythEch]

Wow that looks awesome! More awesome than I have ever imagined! The webchat site looks nice too!

10/10 would use it

I'm not proud of my work but I've remembered that I forgot to show the actual things I talked about.
http://imgur.com/vN1hvfe,iggzjdZ

Off topic:
I'm planning using http://code.google.com/p/dot-net-transitions/ . It's too good for Windows Forms. Lack of animations was driving me insane, especially after I met mobile applications.

[alexhulbert]

I'm going to look over the code for Icew1nd and push the first screen or two of the GUI. Expect a push to today (or tomorrow at most).

---

Off topic:
Before I did Java, I used to be a C# programmer. I could probably help you with WinPie.
When I was 12/13 (not too long ago ;)), I used C# when making an old Virtual Reality Engine (/3RE). I made it out of a broken Wii Remote, my Kinect, some cheap video glasses, an arduino, and some spare infrared lights. It was a really temperamental and buggy, but it was awesome when I mapped my movement to the WSAD keys/mouse and used that with Minecraft.

I want to get an Oculus Rift, but now that Facebook's bought it... I don't know if I want to spend that kind of money.

[alexhulbert]

I'm going to start working on JMobileDevice again. I'll update it to match your changes to afc.py and then I'll work on adding a new python file: springboard.py. This was one of the few files that weren't in pymobiledevice, but I need it. I'm going to duplicate SpringBoard by grabbing all the applications and their icons. I'll find out what page they're on and overlay that on top of the user's background. I'll have the user either drag the icons into an "backup" box or just have them select them normally (like with MultiIconMover-styled check badges). I can't wait to start working on this!