What changes in a jailbroken kernel?

All the “modern” kernel patches are based on comex’s patches.

the main things which are being patched are:

• security.mac.proc_enforce
• cs_enforcement_disable (kernel and AMFI)
• PE_i_can_has_debugger
• vm_map_enter
• vm_map_protect
• …

Oh, and there are sandbox patches too. If you wanna read more about all these patches I suggest you take a look at iOS Hacker’s Handbook.

Edit:
I just came up with a simple idea to check if the device is jailbroken, but I’m not sure if Apple allows the use of these functions:

1. allocate some memory using mach_vm_allocate()

2. change the protection of that page via mach_vm_protect() to VM_PROT_READ | VM_PROT_EXECUTE | VM_PROT_COPY

3. Since the stock iOS doesn’t allow VM_PROT_EXECUTE from inside your app this will fail, check the return value of mach_vm_protect(), when not jailbroken, but succeed if the device is jailbroken.