Improved Original Entry Point Detection Method Based on PinDemonium

KIPS Transactions on Computer and Communication Systems, Vol. 7, No.6, pp.155-164, June 2018
10.3745/KTCCS.2018.7.6.155, Full Text

Abstract

Many malicious programs have been compressed or encrypted using various commercial packers to prevent reverse engineering, So malicious code analysts must decompress or decrypt them first. The OEP (Original Entry Point) is the address of the first instruction executed after returning the encrypted or compressed executable file back to the original binary state. Several unpackers, including PinDemonium, execute the packed file and keep tracks of the addresses until the OEP appears and find the OEP among the addresses. However, instead of finding exact one OEP, unpackers provide a relatively large set of OEP candidates and sometimes OEP is missing among candidates. In other words, existing unpackers have difficulty in finding the correct OEP. We have developed new tool which provides fewer OEP candidate sets by adding two methods based on the property of the OEP. In this paper, we propose two methods to provide fewer OEP candidate sets by using the property that the function call sequence and parameters are same between packed program and original program. First way is based on a function call. Programs written in the C/C++ language are compiled to translate languages into binary code. Compiler-specific system functions are added to the compiled program. After examining these functions, we have added a method that we suggest to PinDemonium to detect the unpacking work by matching the patterns of system functions that are called in packed programs and unpacked programs. Second way is based on parameters. The parameters include not only the user-entered inputs, but also the system inputs. We have added a method that we suggest to PinDemonium to find the OEP using the system parameters of a particular function in stack memory. OEP detection experiments were performed on sample programs packed by 16 commercial packers. We can reduce the OEP candidate by more than 40% on average compared to PinDemonium except 2 commercial packers which are can not be executed due to the anti-debugging technique.


Statistics

Show / Hide Statistics

Statistics (Cumulative Counts from October 15, 2016)

Multiple requests among the same browser session are counted as one view. If you mouse over a chart, the values of data points will be shown.


Cite this paper

[KIPS Transactions Style]
K. G. Min and P. Y. Su, "Improved Original Entry Point Detection Method Based on PinDemonium," KIPS Transactions on Computer and Communication Systems, Vol.7, No.6, pp.155-164, 2018, DOI: 10.3745/KTCCS.2018.7.6.155.

[IEEE Style]
Kim Gyeong Min and Park Yong Su, "Improved Original Entry Point Detection Method Based on PinDemonium," KIPS Transactions on Computer and Communication Systems, vol. 7, no. 6, pp. 155-164, 2018. DOI: 10.3745/KTCCS.2018.7.6.155.

[ACM Style]
Min, K. G. and Su, P. Y. 2018. Improved Original Entry Point Detection Method Based on PinDemonium. KIPS Transactions on Computer and Communication Systems, 7, 6, (2018), 155-164. DOI: 10.3745/KTCCS.2018.7.6.155.