Yaniv's Gems » Comp

Zeroing an Array in Constant Time

yaniv — Thu, 20 Jan 2011 10:04:41 +0000

This one is a really cool (and practical) riddle!

Implement an “integer array” data structure, that supports the following 3 operations:

Init(int n, int k) – initialize the array to be of size n and with all cells set to the value k.

Get(int i) – return the value at cell i.

Set(int i, int k) – set the value of cell i to k.

The catch – all 3 operations should take constant time (not amortized, not probabilistic).

Note – you can assume that malloc or new are constant time, but that the returned memory is filled with (adversarial) random.

Thanks to Nadav Sherman for this riddle!

Differing Neighbors

yaniv — Tue, 11 Jan 2011 22:49:11 +0000

Algorithm A

Given an array of N integers, sort the array, and find the 2 consecutive numbers in the sorted array with the maximum difference.
Example – on input [1,7,3,2] output 4 (the sorted array is [1,2,3,7], and the maximum difference is 7-3=4).

Algorithm A runs in O(NlogN) time.

Implement an algorithm identical in function to algorithm A, that runs in O(N) time.

Thanks to Yossi Richter for this riddle!

The Better Half

yaniv — Wed, 19 May 2010 19:34:33 +0000

A cute and easy algorithmic riddle.

You have an array of N bit strings each of length M. You know that there is at least one element that appears more than N/8 times in the array. Using O(M+log(N)) memory and O(NM) time, find such an element.

An Easier Version

Well, its actually almost exactly the same, but solve the above riddle in case there is an element that appears more than N/2 times in the array. I managed to find 3 distinct solutions to this easier variation, but only one of which generalizes easily.

Thanks Nemo for giving me this riddle!

Find the Duplicate

yaniv — Sat, 13 Oct 2007 19:54:35 +0000

Dany Valevsky gave me this very cool riddle.

You are given a vector of size N, the elements of which are numbers in the range 1,…,N-1. I.e. there is at least one repeating element. Give an algorithm that finds a repeating element (it does not matter which one, in case there are several) with O(N) time complexity and O(1) memory complexity.

NOTE - the time and memory complexities are calculated in integers. I.e. the input is of size N, not N*logN.

That’s it!

Turing Machines in Action

yaniv — Sun, 07 Oct 2007 20:49:18 +0000

In this post I will define turing machines and demonstrate a simple one in action.

Definition of a Turing Machine

A turing machine (TM) description consists of a tuple of 7 objects: a set of states Q, an input alphabet I, a tape alphabet T, a transition function F, a set of accept states Qaccept, a set of reject states Qreject and a start state Qstart.

The following requirements must hold: I is contained in T. ‘ ‘ (the space symbol) is a member of T but not of I. F is a function from QxT->QxTx{1,-1}. Qaccept and Qreject are disjoint subsets of Q. Qstart is a member of Q.

The turing machine itself consists of: its description (as defined above), an infinite tape, a head and a current state.

The tape is an infinite list of symbols from T (it is only infinite to the right, i.e. it has indexes 0, 1, 2, … but no negative indexes). The tape initially contains the input (a string of symbols from I) starting at index 0 followed by infinitely many ’ ‘ (space) symbols. As we required that the ‘ ‘ symbol is not a member of I the TM can detect the end of the input.

The head is simply a pointer to the tape. It initially points to tape position 0.

The current state is a member of Q, and is initialized to Qstart.

Operation

A TM with a description {Q, I, T, F, Qaccept, Qreject, Qstart} is initialized as indicated above. The result of its operation consists of the contents of the tape as well as an additional flag which can be either ACCEPT or REJECT. In the description below, the action accept means setting this flag to the ACCEPT state and halting. The same goes for the reject action.

The TM operates as follows:

If current_state is in Qaccept, accept. If it is in Qreject, reject.
Read a symbol from the tape at the position indicated by the head. Denote it as current_symbol.
Set next_state, next_symbol, next_head_pos to F(current_state, current_symbol).
Write next_symbol to the tape at the current head position.
If next_head_pos is 1 move the head one position to the right.
If next_head_pos is -1 move the head one position to the left. If the head tries to move to the left from position 0, do not move the head.
Set current_state to next_state.

And that’s it!

An Example

The following is the description of a TM that multiplies two integers in unary notation. For example, if its input is of the form ’111*11=’ it will accept with an output tape of ’111*11=111111′.

The code of the machine:

TMUnaryMultiply = { 'description' : """The input must be of the form '11*111='. The output tape will look like '11*111=111111'.""", 'start_state' : 'mark_start', #input verification 'mark_start' : { '1':['verify_input_1','#',1], }, 'verify_input_1' : { '1':['verify_input_1','1',1], '*':['verify_input_*','*',1] }, 'verify_input_*' : { '1':['verify_input_2','1',1] }, 'verify_input_2' : { '1':['verify_input_2','1',1], '=':['verify_input_ ','=',1] }, 'verify_input_ ' : { ' ':['move_head_to_left',' ',-1] }, 'move_head_to_left': { '1':['move_head_to_left','1',-1], '=':['move_head_to_left','=',-1], '*':['move_head_to_left','*',-1], '#':['read_ones','1',-1] # this will try to move the head off the tape }, # actual multiplication 'read_ones' : { '1':['wait_for_*', '#', 1], '*':['halt','*', -1], }, 'wait_for_*' : { '1':['wait_for_*', '1', 1], '*':['copy_ones', '*', 1] }, 'copy_ones' : { '1':['wait_for_=', '#', 1], '=':['move_left2','=', -1], }, 'wait_for_=' : { '1':['wait_for_=','1',1], '=':['wait_for_ ','=',1] }, 'wait_for_ ' : { '1':['wait_for_ ','1',1], ' ':['move_left','1',-1] }, 'move_left': { '1':['move_left','1',-1], '=':['move_left','=',-1], '#':['copy_ones','1', 1] }, 'move_left2': { '1':['move_left2','1',-1], '*':['move_left2','*',-1], '#':['read_ones','1', 1] }, 'halt' : {} }

I think the format of the description is self explanatory (for the python connoisseurs among you, it consists of some nested dictionaries). It can be fed into my Python TM Emulator (the code of which is included at the end of this article). The emulator can also automatically generate a flow graph of the TM. Here is a sample ouput:

Running the emulator (in verbose mode) with the TMUnaryMultiply description above on the input ’111*11=’ results in the following:

Initialing TM [The input must be of the form '11*111='. The output tape will look like '11*111=111111'.]... Running tm with input [111*11=]... V < mark_start>: ['1', '1', '1', '*', '1', '1', '=', ' '] V < verify_input_1>: ['#', '1', '1', '*', '1', '1', '=', ' '] V < verify_input_1>: ['#', '1', '1', '*', '1', '1', '=', ' '] V < verify_input_1>: ['#', '1', '1', '*', '1', '1', '=', ' '] V < verify_input_*>: ['#', '1', '1', '*', '1', '1', '=', ' '] V < verify_input_2>: ['#', '1', '1', '*', '1', '1', '=', ' '] V < verify_input_2>: ['#', '1', '1', '*', '1', '1', '=', ' '] V < verify_input_ >: ['#', '1', '1', '*', '1', '1', '=', ' '] V < move_head_to_left>: ['#', '1', '1', '*', '1', '1', '=', ' '] V < move_head_to_left>: ['#', '1', '1', '*', '1', '1', '=', ' '] V < move_head_to_left>: ['#', '1', '1', '*', '1', '1', '=', ' '] V < move_head_to_left>: ['#', '1', '1', '*', '1', '1', '=', ' '] V < move_head_to_left>: ['#', '1', '1', '*', '1', '1', '=', ' '] V < move_head_to_left>: ['#', '1', '1', '*', '1', '1', '=', ' '] V < move_head_to_left>: ['#', '1', '1', '*', '1', '1', '=', ' '] V < read_ones>: ['1', '1', '1', '*', '1', '1', '=', ' '] V < wait_for_*>: ['#', '1', '1', '*', '1', '1', '=', ' '] V < wait_for_*>: ['#', '1', '1', '*', '1', '1', '=', ' '] V < wait_for_*>: ['#', '1', '1', '*', '1', '1', '=', ' '] V < copy_ones>: ['#', '1', '1', '*', '1', '1', '=', ' '] V < wait_for_=>: ['#', '1', '1', '*', '#', '1', '=', ' '] V < wait_for_=>: ['#', '1', '1', '*', '#', '1', '=', ' '] V < wait_for_ >: ['#', '1', '1', '*', '#', '1', '=', ' '] V < move_left>: ['#', '1', '1', '*', '#', '1', '=', '1'] V < move_left>: ['#', '1', '1', '*', '#', '1', '=', '1'] V < move_left>: ['#', '1', '1', '*', '#', '1', '=', '1'] V < copy_ones>: ['#', '1', '1', '*', '1', '1', '=', '1'] V < wait_for_=>: ['#', '1', '1', '*', '1', '#', '=', '1'] V < wait_for_ >: ['#', '1', '1', '*', '1', '#', '=', '1'] V < wait_for_ >: ['#', '1', '1', '*', '1', '#', '=', '1', ' '] V < move_left>: ['#', '1', '1', '*', '1', '#', '=', '1', '1'] V < move_left>: ['#', '1', '1', '*', '1', '#', '=', '1', '1'] V < move_left>: ['#', '1', '1', '*', '1', '#', '=', '1', '1'] V < copy_ones>: ['#', '1', '1', '*', '1', '1', '=', '1', '1'] V < move_left2>: ['#', '1', '1', '*', '1', '1', '=', '1', '1'] V < move_left2>: ['#', '1', '1', '*', '1', '1', '=', '1', '1'] V < move_left2>: ['#', '1', '1', '*', '1', '1', '=', '1', '1'] V < move_left2>: ['#', '1', '1', '*', '1', '1', '=', '1', '1'] V < move_left2>: ['#', '1', '1', '*', '1', '1', '=', '1', '1'] V < move_left2>: ['#', '1', '1', '*', '1', '1', '=', '1', '1'] V < read_ones>: ['1', '1', '1', '*', '1', '1', '=', '1', '1'] V < wait_for_*>: ['1', '#', '1', '*', '1', '1', '=', '1', '1'] V < wait_for_*>: ['1', '#', '1', '*', '1', '1', '=', '1', '1'] V < copy_ones>: ['1', '#', '1', '*', '1', '1', '=', '1', '1'] V < wait_for_=>: ['1', '#', '1', '*', '#', '1', '=', '1', '1'] V < wait_for_=>: ['1', '#', '1', '*', '#', '1', '=', '1', '1'] V < wait_for_ >: ['1', '#', '1', '*', '#', '1', '=', '1', '1'] V < wait_for_ >: ['1', '#', '1', '*', '#', '1', '=', '1', '1'] V < wait_for_ >: ['1', '#', '1', '*', '#', '1', '=', '1', '1', ' '] V < move_left>: ['1', '#', '1', '*', '#', '1', '=', '1', '1', '1'] V < move_left>: ['1', '#', '1', '*', '#', '1', '=', '1', '1', '1'] V < move_left>: ['1', '#', '1', '*', '#', '1', '=', '1', '1', '1'] V < move_left>: ['1', '#', '1', '*', '#', '1', '=', '1', '1', '1'] V < move_left>: ['1', '#', '1', '*', '#', '1', '=', '1', '1', '1'] V < copy_ones>: ['1', '#', '1', '*', '1', '1', '=', '1', '1', '1'] V < wait_for_=>: ['1', '#', '1', '*', '1', '#', '=', '1', '1', '1'] V < wait_for_ >: ['1', '#', '1', '*', '1', '#', '=', '1', '1', '1'] V < wait_for_ >: ['1', '#', '1', '*', '1', '#', '=', '1', '1', '1'] V < wait_for_ >: ['1', '#', '1', '*', '1', '#', '=', '1', '1', '1'] V < wait_for_ >: ['1', '#', '1', '*', '1', '#', '=', '1', '1', '1', ' '] V < move_left>: ['1', '#', '1', '*', '1', '#', '=', '1', '1', '1', '1'] V < move_left>: ['1', '#', '1', '*', '1', '#', '=', '1', '1', '1', '1'] V < move_left>: ['1', '#', '1', '*', '1', '#', '=', '1', '1', '1', '1'] V < move_left>: ['1', '#', '1', '*', '1', '#', '=', '1', '1', '1', '1'] V < move_left>: ['1', '#', '1', '*', '1', '#', '=', '1', '1', '1', '1'] V < copy_ones>: ['1', '#', '1', '*', '1', '1', '=', '1', '1', '1', '1'] V < move_left2>: ['1', '#', '1', '*', '1', '1', '=', '1', '1', '1', '1'] V < move_left2>: ['1', '#', '1', '*', '1', '1', '=', '1', '1', '1', '1'] V < move_left2>: ['1', '#', '1', '*', '1', '1', '=', '1', '1', '1', '1'] V < move_left2>: ['1', '#', '1', '*', '1', '1', '=', '1', '1', '1', '1'] V < move_left2>: ['1', '#', '1', '*', '1', '1', '=', '1', '1', '1', '1'] V < read_ones>: ['1', '1', '1', '*', '1', '1', '=', '1', '1', '1', '1'] V < wait_for_*>: ['1', '1', '#', '*', '1', '1', '=', '1', '1', '1', '1'] V < copy_ones>: ['1', '1', '#', '*', '1', '1', '=', '1', '1', '1', '1'] V < wait_for_=>: ['1', '1', '#', '*', '#', '1', '=', '1', '1', '1', '1'] V < wait_for_=>: ['1', '1', '#', '*', '#', '1', '=', '1', '1', '1', '1'] V < wait_for_ >: ['1', '1', '#', '*', '#', '1', '=', '1', '1', '1', '1'] V < wait_for_ >: ['1', '1', '#', '*', '#', '1', '=', '1', '1', '1', '1'] V < wait_for_ >: ['1', '1', '#', '*', '#', '1', '=', '1', '1', '1', '1'] V < wait_for_ >: ['1', '1', '#', '*', '#', '1', '=', '1', '1', '1', '1'] V < wait_for_ >: ['1', '1', '#', '*', '#', '1', '=', '1', '1', '1', '1', ' '] V < move_left>: ['1', '1', '#', '*', '#', '1', '=', '1', '1', '1', '1', '1'] V < move_left>: ['1', '1', '#', '*', '#', '1', '=', '1', '1', '1', '1', '1'] V < move_left>: ['1', '1', '#', '*', '#', '1', '=', '1', '1', '1', '1', '1'] V < move_left>: ['1', '1', '#', '*', '#', '1', '=', '1', '1', '1', '1', '1'] V < move_left>: ['1', '1', '#', '*', '#', '1', '=', '1', '1', '1', '1', '1'] V < move_left>: ['1', '1', '#', '*', '#', '1', '=', '1', '1', '1', '1', '1'] V < move_left>: ['1', '1', '#', '*', '#', '1', '=', '1', '1', '1', '1', '1'] V < copy_ones>: ['1', '1', '#', '*', '1', '1', '=', '1', '1', '1', '1', '1'] V < wait_for_=>: ['1', '1', '#', '*', '1', '#', '=', '1', '1', '1', '1', '1'] V < wait_for_ >: ['1', '1', '#', '*', '1', '#', '=', '1', '1', '1', '1', '1'] V < wait_for_ >: ['1', '1', '#', '*', '1', '#', '=', '1', '1', '1', '1', '1'] V < wait_for_ >: ['1', '1', '#', '*', '1', '#', '=', '1', '1', '1', '1', '1'] V < wait_for_ >: ['1', '1', '#', '*', '1', '#', '=', '1', '1', '1', '1', '1'] V < wait_for_ >: ['1', '1', '#', '*', '1', '#', '=', '1', '1', '1', '1', '1'] V < wait_for_ >: ['1', '1', '#', '*', '1', '#', '=', '1', '1', '1', '1', '1', ' '] V < move_left>: ['1', '1', '#', '*', '1', '#', '=', '1', '1', '1', '1', '1', '1'] V < move_left>: ['1', '1', '#', '*', '1', '#', '=', '1', '1', '1', '1', '1', '1'] V < move_left>: ['1', '1', '#', '*', '1', '#', '=', '1', '1', '1', '1', '1', '1'] V < move_left>: ['1', '1', '#', '*', '1', '#', '=', '1', '1', '1', '1', '1', '1'] V < move_left>: ['1', '1', '#', '*', '1', '#', '=', '1', '1', '1', '1', '1', '1'] V < move_left>: ['1', '1', '#', '*', '1', '#', '=', '1', '1', '1', '1', '1', '1'] V < move_left>: ['1', '1', '#', '*', '1', '#', '=', '1', '1', '1', '1', '1', '1'] V < copy_ones>: ['1', '1', '#', '*', '1', '1', '=', '1', '1', '1', '1', '1', '1'] V < move_left2>: ['1', '1', '#', '*', '1', '1', '=', '1', '1', '1', '1', '1', '1'] V < move_left2>: ['1', '1', '#', '*', '1', '1', '=', '1', '1', '1', '1', '1', '1'] V < move_left2>: ['1', '1', '#', '*', '1', '1', '=', '1', '1', '1', '1', '1', '1'] V < move_left2>: ['1', '1', '#', '*', '1', '1', '=', '1', '1', '1', '1', '1', '1'] V < read_ones>: ['1', '1', '1', '*', '1', '1', '=', '1', '1', '1', '1', '1', '1'] accept

Finally, here is the code of my Python TM Emulator:

class TM: def __init__(self, states, accept_states, reject_states): print 'Initialing TM [%s]...' % (states['description']) self.states = states self.accept_states = accept_states self.reject_states = reject_states self.halt_states = accept_states + reject_states def run(self, input,verbose=False): head_pos = 0 current = self.states['start_state'] tape = [x for x in input]+[" "] while current not in self.halt_states: if verbose: print ' '*(22+1+1+1+5*(head_pos)+1)+'V' print '<%20s>: %s' % (current, tape) try: next_state, next_symbol, head_movement = self.states[current][tape[head_pos]] except KeyError: return "reject" tape[head_pos] = next_symbol head_pos += head_movement if head_pos < 0: head_pos = 0 elif head_pos == len(tape): tape.append(" ") current = next_state print ''.join(tape) if current in self.accept_states: return 'accept' else: assert(current in self.reject_states) return 'reject'

And that of a simple test method:

def test(): mul = TM(TMUnaryMultiply, ["halt"], [""]) inputs = ['bad_input','111*11=','1*1=','*1=','1*=','11111111*1111111='] for input in inputs: print 'Running tm with input [%s]...' % (input) print mul.run(input)

A sample output:

>>> test() Initialing TM [The input must be of the form '11*111='. The output tape will look like '11*111=111111'.]... Running tm with input [bad_input]... reject Running tm with input [111*11=]... 111*11=111111 accept Running tm with input [1*1=]... 1*1=1 accept Running tm with input [*1=]... reject Running tm with input [1*=]... reject Running tm with input [11111111*1111111=]... 11111111*1111111=11111111111111111111111111111111111111111111111111111111 accept

A second, more interesting TM performs binary addition:

TMBinaryAddReverse = { 'description' : """The input must be of the form '1010+0010=' (i.e. in reversed binary notation, operands of same size padded with an extra 0). The output will look like '1010+0010=1110'.""", 'start_state' : 'mark_start_carry_0', 'halt': {}, 'mark_start_carry_0' : { '1':['sum_1','#',1], '0':['sum_0','#',1], '+':['halt','+',1] }, 'mark_start_carry_1' : { '1':['sum_2','#',1], '0':['sum_1','#',1], '+':['halt','+',1] }, 'sum_0' : { '1':['sum_0','1',1], '0':['sum_0','0',1], '+':['sum_0_get_next','+',1], '=':['write_0','=',1], }, 'sum_1' : { '1':['sum_1','1',1], '0':['sum_1','0',1], '+':['sum_1_get_next','+',1], '=':['write_1','=',1], }, 'sum_2' : { '1':['sum_2','1',1], '0':['sum_2','0',1], '+':['sum_2_get_next','+',1], '=':['write_2','=',1], }, 'sum_3' : { '1':['sum_3','1',1], '0':['sum_3','0',1], '=':['write_3','=',1], }, 'sum_1_get_next': { '#':['sum_1_get_next','#',1], '1':['sum_2','#',1], '0':['sum_1','#',1], }, 'sum_0_get_next': { '#':['sum_0_get_next','#',1], '1':['sum_1','#',1], '0':['sum_0','#',1], }, 'sum_2_get_next': { '#':['sum_2_get_next','#',1], '1':['sum_3','#',1], '0':['sum_2','#',1], }, 'write_0': { '1':['write_0','1',1], '0':['write_0','0',1], ' ':['move_left_carry_0','0',-1], }, 'write_1': { '1':['write_1','1',1], '0':['write_1','0',1], ' ':['move_left_carry_0','1',-1], }, 'write_2': { '1':['write_2','1',1], '0':['write_2','0',1], ' ':['move_left_carry_1','0',-1], }, 'write_3': { '1':['write_3','1',1], '0':['write_3','0',1], ' ':['move_left_carry_1','1',-1], }, 'move_left_carry_0': { '0':['move_left_carry_0','0',-1], '1':['move_left_carry_0','1',-1], '=':['move_left_carry_0','=',-1], '#':['move_left_carry_0','#',-1], '+':['move_left_carry_0_part_2','+',-1], }, 'move_left_carry_1': { '0':['move_left_carry_1','0',-1], '1':['move_left_carry_1','1',-1], '=':['move_left_carry_1','=',-1], '#':['move_left_carry_1','#',-1], '+':['move_left_carry_1_part_2','+',-1], }, 'move_left_carry_0_part_2': { '0':['move_left_carry_0_part_2','0',-1], '1':['move_left_carry_0_part_2','1',-1], '#':['mark_start_carry_0','#',1], }, 'move_left_carry_1_part_2': { '0':['move_left_carry_1_part_2','0',-1], '1':['move_left_carry_1_part_2','1',-1], '#':['mark_start_carry_1','#',1], }, }

Its graph is included here for your convenience (click to enlarge):

And a sample run:

>>> mul = TM(TMBinaryAddReverse, ["halt"], []) Initialing TM [The input must be of the form '1010+0010=' (i.e. in reversed binary notation, operands of same size padded with an extra 0). The output will look like '1010+0010=1110'.]... >>> mul.run('1110+1100=',True) V : ['1', '1', '1', '0', '+', '1', '1', '0', '0', '=', ' '] V : ['#', '1', '1', '0', '+', '1', '1', '0', '0', '=', ' '] V : ['#', '1', '1', '0', '+', '1', '1', '0', '0', '=', ' '] V : ['#', '1', '1', '0', '+', '1', '1', '0', '0', '=', ' '] V : ['#', '1', '1', '0', '+', '1', '1', '0', '0', '=', ' '] V : ['#', '1', '1', '0', '+', '1', '1', '0', '0', '=', ' '] V : ['#', '1', '1', '0', '+', '#', '1', '0', '0', '=', ' '] V : ['#', '1', '1', '0', '+', '#', '1', '0', '0', '=', ' '] V : ['#', '1', '1', '0', '+', '#', '1', '0', '0', '=', ' '] V : ['#', '1', '1', '0', '+', '#', '1', '0', '0', '=', ' '] V : ['#', '1', '1', '0', '+', '#', '1', '0', '0', '=', ' '] V : ['#', '1', '1', '0', '+', '#', '1', '0', '0', '=', '0'] V : ['#', '1', '1', '0', '+', '#', '1', '0', '0', '=', '0'] V : ['#', '1', '1', '0', '+', '#', '1', '0', '0', '=', '0'] V : ['#', '1', '1', '0', '+', '#', '1', '0', '0', '=', '0'] V : ['#', '1', '1', '0', '+', '#', '1', '0', '0', '=', '0'] V : ['#', '1', '1', '0', '+', '#', '1', '0', '0', '=', '0'] V : ['#', '1', '1', '0', '+', '#', '1', '0', '0', '=', '0'] V : ['#', '1', '1', '0', '+', '#', '1', '0', '0', '=', '0'] V : ['#', '1', '1', '0', '+', '#', '1', '0', '0', '=', '0'] V : ['#', '1', '1', '0', '+', '#', '1', '0', '0', '=', '0'] V : ['#', '1', '1', '0', '+', '#', '1', '0', '0', '=', '0'] V : ['#', '#', '1', '0', '+', '#', '1', '0', '0', '=', '0'] V : ['#', '#', '1', '0', '+', '#', '1', '0', '0', '=', '0'] V : ['#', '#', '1', '0', '+', '#', '1', '0', '0', '=', '0'] V : ['#', '#', '1', '0', '+', '#', '1', '0', '0', '=', '0'] V : ['#', '#', '1', '0', '+', '#', '1', '0', '0', '=', '0'] V : ['#', '#', '1', '0', '+', '#', '#', '0', '0', '=', '0'] V : ['#', '#', '1', '0', '+', '#', '#', '0', '0', '=', '0'] V : ['#', '#', '1', '0', '+', '#', '#', '0', '0', '=', '0'] V : ['#', '#', '1', '0', '+', '#', '#', '0', '0', '=', '0'] V : ['#', '#', '1', '0', '+', '#', '#', '0', '0', '=', '0', ' '] V : ['#', '#', '1', '0', '+', '#', '#', '0', '0', '=', '0', '1'] V : ['#', '#', '1', '0', '+', '#', '#', '0', '0', '=', '0', '1'] V : ['#', '#', '1', '0', '+', '#', '#', '0', '0', '=', '0', '1'] V : ['#', '#', '1', '0', '+', '#', '#', '0', '0', '=', '0', '1'] V : ['#', '#', '1', '0', '+', '#', '#', '0', '0', '=', '0', '1'] V : ['#', '#', '1', '0', '+', '#', '#', '0', '0', '=', '0', '1'] V : ['#', '#', '1', '0', '+', '#', '#', '0', '0', '=', '0', '1'] V : ['#', '#', '1', '0', '+', '#', '#', '0', '0', '=', '0', '1'] V : ['#', '#', '1', '0', '+', '#', '#', '0', '0', '=', '0', '1'] V : ['#', '#', '1', '0', '+', '#', '#', '0', '0', '=', '0', '1'] V : ['#', '#', '1', '0', '+', '#', '#', '0', '0', '=', '0', '1'] V : ['#', '#', '#', '0', '+', '#', '#', '0', '0', '=', '0', '1'] V : ['#', '#', '#', '0', '+', '#', '#', '0', '0', '=', '0', '1'] V : ['#', '#', '#', '0', '+', '#', '#', '0', '0', '=', '0', '1'] V : ['#', '#', '#', '0', '+', '#', '#', '0', '0', '=', '0', '1'] V : ['#', '#', '#', '0', '+', '#', '#', '0', '0', '=', '0', '1'] V : ['#', '#', '#', '0', '+', '#', '#', '#', '0', '=', '0', '1'] V : ['#', '#', '#', '0', '+', '#', '#', '#', '0', '=', '0', '1'] V : ['#', '#', '#', '0', '+', '#', '#', '#', '0', '=', '0', '1'] V : ['#', '#', '#', '0', '+', '#', '#', '#', '0', '=', '0', '1'] V : ['#', '#', '#', '0', '+', '#', '#', '#', '0', '=', '0', '1', ' '] V : ['#', '#', '#', '0', '+', '#', '#', '#', '0', '=', '0', '1', '0'] V : ['#', '#', '#', '0', '+', '#', '#', '#', '0', '=', '0', '1', '0'] V : ['#', '#', '#', '0', '+', '#', '#', '#', '0', '=', '0', '1', '0'] V : ['#', '#', '#', '0', '+', '#', '#', '#', '0', '=', '0', '1', '0'] V : ['#', '#', '#', '0', '+', '#', '#', '#', '0', '=', '0', '1', '0'] V : ['#', '#', '#', '0', '+', '#', '#', '#', '0', '=', '0', '1', '0'] V : ['#', '#', '#', '0', '+', '#', '#', '#', '0', '=', '0', '1', '0'] V : ['#', '#', '#', '0', '+', '#', '#', '#', '0', '=', '0', '1', '0'] V : ['#', '#', '#', '0', '+', '#', '#', '#', '0', '=', '0', '1', '0'] V : ['#', '#', '#', '0', '+', '#', '#', '#', '0', '=', '0', '1', '0'] V : ['#', '#', '#', '0', '+', '#', '#', '#', '0', '=', '0', '1', '0'] V : ['#', '#', '#', '#', '+', '#', '#', '#', '0', '=', '0', '1', '0'] V : ['#', '#', '#', '#', '+', '#', '#', '#', '0', '=', '0', '1', '0'] V : ['#', '#', '#', '#', '+', '#', '#', '#', '0', '=', '0', '1', '0'] V : ['#', '#', '#', '#', '+', '#', '#', '#', '0', '=', '0', '1', '0'] V : ['#', '#', '#', '#', '+', '#', '#', '#', '0', '=', '0', '1', '0'] V : ['#', '#', '#', '#', '+', '#', '#', '#', '#', '=', '0', '1', '0'] V : ['#', '#', '#', '#', '+', '#', '#', '#', '#', '=', '0', '1', '0'] V : ['#', '#', '#', '#', '+', '#', '#', '#', '#', '=', '0', '1', '0'] V : ['#', '#', '#', '#', '+', '#', '#', '#', '#', '=', '0', '1', '0'] V : ['#', '#', '#', '#', '+', '#', '#', '#', '#', '=', '0', '1', '0', ' '] V : ['#', '#', '#', '#', '+', '#', '#', '#', '#', '=', '0', '1', '0', '1'] V : ['#', '#', '#', '#', '+', '#', '#', '#', '#', '=', '0', '1', '0', '1'] V : ['#', '#', '#', '#', '+', '#', '#', '#', '#', '=', '0', '1', '0', '1'] V : ['#', '#', '#', '#', '+', '#', '#', '#', '#', '=', '0', '1', '0', '1'] V : ['#', '#', '#', '#', '+', '#', '#', '#', '#', '=', '0', '1', '0', '1'] V : ['#', '#', '#', '#', '+', '#', '#', '#', '#', '=', '0', '1', '0', '1'] V : ['#', '#', '#', '#', '+', '#', '#', '#', '#', '=', '0', '1', '0', '1'] V : ['#', '#', '#', '#', '+', '#', '#', '#', '#', '=', '0', '1', '0', '1'] V : ['#', '#', '#', '#', '+', '#', '#', '#', '#', '=', '0', '1', '0', '1'] V : ['#', '#', '#', '#', '+', '#', '#', '#', '#', '=', '0', '1', '0', '1'] V : ['#', '#', '#', '#', '+', '#', '#', '#', '#', '=', '0', '1', '0', '1'] ####+####=0101 'accept'

I think this machine can be simplified by writing the carry directly to the tape (i.e. writing a ’0′, a ’1′ or a ’2′ to the result part of the tape, and later converting the ’2′s to ’1′ and ’0′ as appropriate).

If anyone of you manages to generate an equivalent machine with a shorter description (less states) please post it!

Seam Carving

yaniv — Mon, 27 Aug 2007 20:39:38 +0000

I recently watched this interesting video by Shai Avidan and Ariel Shamir from MERL. They developed an extremely cool image resizing technique called seam carving. They explain all about it in this article.

One of the coolest things about their idea is that is it easy to implement. I implemented a semi-optimized version of their algorithm in a couple of hours of coding. All the images in this post were generated by my code.

The following image of a pagoda will assist me in demonstrating the ideas to follow:

First lets discuss the two image resizing methods currently used, namely scaling and cropping. Scaling consists of uniformly resizing the image. If for example we were to reduce an image 100 pixels wide to an image 50 pixels wide, we could simply average every two neighbouring pixels (a generalization of this technique can be applied even when the image’s new width does not divide the image’s original width, and of course to resizing both the width and the height of the image).

Scaling is efficient and, in some sense, you do not lose a lot of information from the original image. There are two big problems with scaling however:

If the new dimensions of the image are not proportionate to the original dimensions, the scaled image is distorted.
The method does not take into account the contents of the image at all (I will clarify this point shortly).

An example of the pagoda image scaled:

A second widely used form of image resizing is cropping. Cropping consists of simply selecting a sub image of the desired proportions from the original image – i.e. removing the borders of the image. An important question now arises: how do we select which box of the image to keep (or equivalently, how do we select which borders to remove)?

Lets say we assign a weight to each of the pixels of the image, representing its importance. Then selecting the cropping box amounts to simply selecting the box containing the pixels whose sum of weights is highest. We will refer to the weights of the pixels as the energy of the pixels. So optimal cropping would select the box with the highest energy.

Now, all we are left to do is assign the weights to the pixels. Following Avidan and Shamir, we can use simple gradient magnitude as our energy function. Gradient magnitude simply means calculating the gradient at each pixel (regarding the image as a function from R2 to R) and taking its magnitude at each pixel. Here is the result of applying gradient magnitude on the pagoda image:

The perceptive reader would have noticed that a color image is usually a function from R2 to R3 and not to R (as there are usually three independent color channels). There are several ways to calculate the gradient magnitude in this case. The one I used in my code is to calculate the magnitudes of the three gradients separately and average the results. Note that averaging before calculating the gradients is a bad idea as a lot of information is lost in the process!

Note that the gradient magnitude of a pixel constitutes some measure of how likely that pixel is to be a border pixel (and thus more important).

Using all of this, an optimal cropping of the pagoda image is shown here:

The cropping is optimal in the sense that this is the sub-image of the given dimensions containing the most energy.

Note that all the water from the bottom of the image, and the flowers from the top are missing, as well as the left side of the white house on the left of the pagoda.

Now, unlike the scaling method, the cropping technique just presented takes into account the contents of the image. But what if the image contained two pagodas, one at each side of the image? Cropping would only be able to select one of them.

So what we are looking for is some way to automatically remove the uninteresting parts in the middle of the image, and bring the interesting bits closer together.

A naive solution, similar to regular cropping, would be to remove the columns of the image with the least amount of energy not necessarily from the border. This technique causes severe artifacts.

Avidan’s and Shamir’s seam carving technique is an improvement of this. They define a vertical seam as an 8-connected path from top-to-bottom of the image containing one pixel in each row. 8-connectedness means that if pixel (x,y) is in the vertical seam, then exactly one of the pixels (x+1, y+1), (x, y+1) or (x-1, y) is in the vertical seam as well (unless of course y = the height of the image). A horizontal seam is defined similarly.

Now instead of deleting the column (row) with the least energy, they suggest deleting the vertical (horizontal) seam with the least energy. The seam with the least energy can be easily found using dynamic programming.

The first seam to be removed in the pagoda image is depicted below:

The process of reducing an image’s size via repeated deletions of the least important seams is called seam carving. The result of applying some seam carving to the pagoda image is shown here:

Note that although some artifacts are present in the image, it is the best among all the reduced images (at least – a mon avis).

A problem with seam carving, compared to scaling, is efficiency. I have some algorithmic ideas that I think might substantially reduce computation costs. I will let you know when I will have time to test them.

Another example is shown. The original image:

Optimal cropping:

Scaling:

And finally, seam carving:

And just to convince you that the method doesn’t only work on pagodas:

And the carved version:

SIMD (Fire)Works!

yaniv — Sun, 12 Aug 2007 00:28:47 +0000

When I started yaniv.leviathanonline.com, I declared it to be a site about “Mathematics and Hacking”. As of now, most of its content is Mathematical. I thought it was time for a little change…

Those of you with no interest in assembly or low-level programming are welcome to skip this post entirely, although expanding your horizons is never a bad idea!

A few years ago I gave a week-long x86 assembly course for some fellow programmers. As I wanted them to learn how to use the Pentium’s SIMD extensions, I asked them to program a simple fireworks demo and try to apply some filters to it using MMX. In this article I will detail my solution to the SIMD fireworks exercise.

The following paragraph is intended for those of you without assembly knowledge. All of you assembly masters are welcome to skip to the fun stuff below!

Some References

Well, for starters I recommend reading the ”The Art of Assembly Language Programming” (AoA) which covers a lot of the basics. And yes – Assembly Programming isan Art (maybe more so than programming in other languages). Next, you should also read Intel’s manuals: Volume 1: Basic Architecture, Volume 2A: Instruction Set Reference, A-M, Volume 2B: Instruction Set Reference, N-Z, Volume 3A: System Programming Guide, Volume 3B: System Programming Guide and Architectures Optimization Reference Manual. Yes, I know its a lot of reading but they are really interesting and you don’t have to read the whole thing in order (although I recommend it – so many interesting facts lye in those pages). Finally, if you are going to develop in assembly (e.g. assemble the source included in this article) I recommend using NASM, as it is the simplest and cleanest assembler (although some people prefer MASM, which is kind of similar, but a bit more clumsy). If you are going to do some 16-bit DOS programming, you will find Ralf Brown’s Interrupt List (RBIL) a useful tool.

Throughout this article I will assume basic assembly knowledge (the first chapters of AoA will more than cover everything).

The Buzzwords

SIMD stands for Single Instruction Multiple Data. It is clear that most algorithms require repeating the same operation over and over again (each time on different sets of data). From this observation stemmed the idea of a vector computer or SIMD. We will shortly see that indeed our fireworks program will repeat the same operations many times and so we will be using the Pentium’s SIMD capabilities for that purpose.

MMX was Intel’s first attempt at bringing SIMD capabilities to the Pentium. MMX does not stand for anything, but it has been offered several meanings since its appearance (such as Multimedia Extensions). MMX adds 8 new registers (mm0,…,mm7) to the CPU. These are 64-bit registers. The main advantage of these registers (compared to the x86 general purpose registers) is that the MMX registers can function as vector registers. This means that each of these registers can represent one 64-bit integer value, two 32-bit integer values, four 16-bit integer values or eight 8-bit integer values. For example, the command:

psubusb mm0, mm1

is essentially equivalent to 8 byte subtractions (it stands for Packed SUBtract Unsigned Saturated Byte).

Note that in order to be backward compatible with existing OS’s (mainly Windows) Intel aliased the MMX registers to the FPU registers. This means that when accessing one of them the contents of the other one are altered as well. This allowed the OS’s to keep using their old context switching mechanisms.

SSE stands for Streaming SIMD Extensions. This was Intel’s second attempt at bringing SIMD to the PC (in response to AMD’s 3DNow!). SSE added many instructions to the instruction set as well as 16 128-bit vector registers (xmm0,…,xmm15). Unlike the 8 MMX registers, the SSE registers can contain four 32-bit floating point values. As these registers are not aliased to any existing registers (and so require special care by the OS) they are disabled by default and are only turned on when the OS explicitly commands it.

Mode 13h

The easiest to program DOS graphics mode is probably mode 13h, which we shall therefore use. In mode 13h, the screen is represented as a 320*200 byte buffer (located in address 0xA000). Each byte in the buffer is an index to a 256 bytes long palette. When mode 13h is enabled, simply writing bytes memory in the range 0xA000-0x19A00 (0x19A0 = 0xA000 + 0xFA00, 0xFA00 = 320*200) will draw pixels on the screen. You should not draw directly to this buffer though, as when the video card is drawing the screen, it scans through this buffer, thereby stalling your write commands. What you should do, is allocate an off-screen buffer and render to this buffer and once in a frame spill the contents of this buffer as quickly as possible to the memory block starting at 0xA000. We will get to this later.

Entering mode 13h is very easy, and is done by calling BIOS function 0×10, as such:

mov ax, 0x13 int 0x10

Calling BIOS functions is done through raising an interrupt. The parameters are passed in the registers (for more information see RBIL).

The default video mode for DOS is mode 3h. It is the mode of choice for displaying text. So upon program exit we should restore the video mode to mode 3h, as such:

mov ax, 3 int 0x10

The Palette

Now, in order to render properly colored fireworks, we need to have a decent color palette. In mode 13h the palette is set by outputting the palette index number to port 0x3c8 and then outputting 3 color values (for the red, green and blue components) to port 0x3c9. Note that if the entire palette is to be set in one go, only index 0 needs to be outputted to port 0x3c8, as it is incremented automatically after three writes to port 0x3c9. Each of the 3 color components is represented by a 7 bit value, it is a number in the range 0-63, where a value of 0 means that the component is not present at all in the color, and a value of 63 means that the component is present with full opacity in the color. So the color pure green will be represented by the tuple (0,63,0). The palette we shall use will consist of a simple linear interpolation from black to red to white. Index 0 will be total black, index 128 will be total red and index 255 will be total white.

A simple program that sets such a palette and then draws it to the screen can be found here (its source is here) Our fireworks program will use most of this code. Its output is:

The Basic Fireworks Program

Enough with the introductions – lets get to business!

The design for the fireworks program is very simple. We will manage a buffer of particles, each having the following qualities: x, y, x_speed, y_speed and color (although this version uses a constant white color for all particles, which looks better). Each particle’s position will be updated according to its speed, which will in turn be updated according to gravitation.

The full source of the program can be found here. The program itself can be found here. It begins with some defines (you can play with the number of particles by changing the NUM_PARTS define). I then allocate a buffer big enough to contain a copy of the entire screen (we will draw to this buffer, so as to avoid the stalling inherent in writing to the video memory itself). We access the buffer though the es register.

We then enter mode 13h, set the palette and zero the buffer.

We then use a little very useful trick. We call the RDTSC instruction, which reads the CPU’s 64-bit time-stamp register (this time-stamp register is zeroed upon CPU initialization – i.e. when you turn on your computer – and is incremented after every instruction! What do you say, is there a chance of overflow?) into registers EDX:EAX. As NASM does not support the RDTSC instruction I coded it as:

db 0x0f, 0x31

This trick is very useful, as when writing C++ programs for the PC, it can be used to generate very accurate time-stamps very efficiently. The following code will do the trick in Visual Studio:

int RDTSC() { __asm _emit 0x0f; __asm _emit 0x31; }

Lets get back to the fireworks program. We use this trick to generate a random seed for the random number generator (RNG). This RNG will later be used to initialize the particles.

Next, is the init_particle function, which as its name implies, initializes a particle’s properties. In order to make the particles look like fireworks I initialize the position of bunches of particles to the same value (using the global_x and global_y variables). The init_particle function accepts a single argument (a pointer to a particle) passed through the bp register.

The MAIN function updates the particles (and calls init_particle when needed) and then draws the particles to our off-screen buffer.

After all of the particles are drawn, the MAIN function calls two functions which apply filters on our buffer, before displaying it on the screen (by copying our video buffer to address 0xA000).

The Filters

These filters are very important (and are implemented using MMX!). There are three: mmx_shade, mmx_blur and mmx_blur_right.

The particles are drawn by the MAIN function with a solid white color (i.e. an index of 255). Instead of wiping the screen after each frame, we call the mmx_shade function which decreases the value of all the pixels on the screen by 1. Its code is very simple:

mmx_shade: mov cx, 320*200/8 xor di, di movq mm1, [sub_mask] .lop: movq mm0, [es:di] psubusb mm0, mm1 movq [es:di], mm0 add di, 8 loop .lop ret

Where,

sub_mask: dd 0x01010101, 0x01010101

Note how simple this is! First of all we gain a big speed factor as we are processing 8 pixels in each iteration. Even more importantly we use the extremely convenient saturated subtraction instruction. Saturated subtraction means that if the result of the subtraction is negative, then instead of getting the value mod 256, we get 0!

Running the fireworks program with only the mmx_shade filter on, results in this:

Not very impressive yet, but it does look like fireworks! (at least when animated…)

In order to make our program much nicer, we use two more blurring filters. The mmx_blur function averages the value of all pixels with that of their 4 immediate neighbours (giving twice more weight to the value already in the current pixel). Its code is:

mmx_blur: mov cx, (320*200-330*2)/8 mov di, 328 .lop: movq mm0, [es:di] movq mm1, [es:di+1] movq mm2, [es:di-1] movq mm3, mm0 movq mm4, [es:di-320] movq mm5, [es:di+320] pavgb mm0, mm1 pavgb mm3, mm2 pavgb mm4, mm5 pavgb mm3, mm4 pavgb mm0, mm3 movq [es:di], mm0 add di, 8 loop .lop ret

Notice that we skip the first and last lines (plus several extra bytes) in order to keep the function as simple as possible. Those of you with sharper eyes will observe a small issue with this function. In order to properly perform these averages, we need to use a third buffer. As we are scanning through our buffer, we use the results of previous blurs as inputs to the following ones, instead of using the original contents of the buffer. This is not very noticeable, and as it saves us the need to allocate a third buffer it is worth it.

Running the program with the mmx_blur filter applied, as well as the mmx_shade filter, results in this:

Much nicer! Finally we have the mmx_blur_right function, which is a slight variation of mmx_blur, that gives the fireworks comet like appearance. Its code is self-explanatory:

mmx_blur_right: mov cx, (320*200-330*2)/8 mov di, 328 .lop: movq mm0, [es:di] movq mm1, [es:di+1] movq mm2, [es:di+320] movq mm3, [es:di+321] pavgb mm0, mm1 pavgb mm3, mm2 pavgb mm0, mm3 movq [es:di], mm0 add di, 8 loop .lop ret

The MAIN program switches between the two blur functions once in a while. The result of applying the mmx_blur_right function is:

Exercises to the Reader

Change the three filter functions so that they use the 16-byte SSE registers, thus gaining a factor of 2 on these functions’ running time. This is very easy!
Add another offscreen buffer and correctly implement the blurring.
Make the program shorter/cleaner/nicer. Play with the parameters. Change the palette. Enjoy!

I hope that you now have a decent understanding of SIMD and how to use it when programming the Pentium!

PS – in case you want to assemble the source use the following command:

nasmw.exe fireworks.asm -o fireworks.com

I’m Back (and so is Phrack)!

yaniv — Thu, 19 Jul 2007 01:42:26 +0000

Hi everyone!

As you may have noticed, yaniv.leviathanonline.com has not been updated for a while (exactly one month, pour être précis). This was largely due to my intense exam period.

First, I want to thank you for all the interesting ideas you sent me. This extended period of not updating my site resulted in an extremely long idea-list – so stay tuned!

Now, apart from me, Phrack, which was also on a long vacation, is now back.

I must admit that the new issue #64 does not appear to contain anything ground-breaking, but it deserves a quick read (if only for times past).

For those of you that don’t know Phrack, I can only recommend that you browse its outstanding articles’ archive since the very first issue on November 17, 1985 (I was exactly 3 years and one day old then…).

Should you not be interested in hacking or general anarchy, and thus choose not to read all of the issues, at least be sure not to miss the famous Phrack #49′s Smashing The Stack For Fun And Profit. To get an idea of the Phrack spirit, I also recommend reading the Hacker’s Manifesto and Screwing over your local McDonald’s.

See you soon!

Secure Computing – Part I

yaniv — Thu, 07 Jun 2007 16:51:00 +0000

This article is the first among a series of articles describing the subject of Secure Computing. The material was mainly taken from a Cryptography course by Amos Fiat.

Say we have a function F(x1,x2,…,xn)->(y1,y2,…,yn) that takes n inputs and has n outputs. Me and n-1 of my friends (i.e. we are n people in total) have secret numbers. Lets number the people p1,…,pn and the secret numbers x1,…,xn (i.e. person pi knows secret number xi). We want to perform a process after which person pi will know yi, where yi is the i’th output of F computed on the xi’s, but he must not know anything else. Let me give you an example:

Say that there are 3 people, each having a monthly salary of xi dollars (for i=1,2,3). They want person number 1 to know the sum of the salaries, person 2 to know the product of the salaries and person 3 to know the maximum of the salaries. But they do not want to reveal their salaries to each other!

A simple solution would be to find a new and neutral person that all of the men trust. They would then reveal to him all of their salaries, one by one, and he will perform the calculation and tell them the results. The goal of this series of articles is to solve this problem without using such a trusted party.

Before we begin to think of a solution, lets notice some inherent problems:

The men can lie about their salaries. Since it is assumed that only person pi knows xi, he can lie about it (i.e. input another number, say xi’ instead of xi). There is obviously nothing that can be done about it. We will therefore assume all the men (while they may try to cheat at other parts of the protocol) will input their true numbers to it.
The function itself may leak back information about the secret numbers of the other people. For example, if there are only 2 people, and y1 is the sum of the xi’s and y2 is their product, then after the process each of the men will know the secret number of the other (as the function is reversible). This is simply not considered a problem, as our goal was to do without the trusted party, and if for some reason the function is reversible (or semi-reversible) then the same problem will arise with the trusted party protocol, which is what we are trying to emulate.

The solution will consist of several parts (which will be dealt with in the other parts of this topic):

Performing a Zero Knowledge Proof (ZKP) of a generic NPC problem.
The “Oblivious Transfer” protocol.
Transforming a general function f(x1,…,xn) to a circuit consisting only of XOR gates and AND gates.
Solving our problem for the specific cases where our function is a XOR gate or an AND gate.

So now, after we have set our goals, lets dive in to the the first step:

Zero Knowledge Proof of an NPC Problem

At this point, I rather not formally define the exact meaning of a ZKP. I will instead try to explain is intuitively. Say I know the answer to a complicated puzzle and you don’t. The process of me convincing you that I know the answer, without you learning anything about it, is called a Zero Knowledge Proof.

Let me give you this neat example (taken from Applied Kid Cryptography). Say we are playing the game of Where’s Waldo? with this picture:

After minutes of contemplation, I suddenly find him! The following dialog then takes place between us:

Me: “Yes! I found him! Wow, he was really well hidden.”

You: “I don’t believe you! Its too hard! You must be lying!”

Me: “No, seriously. He’s right there, next to the…”

You: “Go on – Where is he?!”

Me: “Well, I do not want to ruin this for you… This is such a cool puzzle…”

You: “I knew it! You are lying – you haven’t found him.”

The situation becomes frustrating for me! How can I prove to you that I know where Waldo is without revealing his location?

I can give you a general fact about his location (for example, that he is near the center of the picture and not near the borders) and later when you will find him too, you will be able to verify that I was indeed right.

There are obvious drawbacks with this method:

While not revealing exactly where Waldo is, a lot of information is still leaked (you now know not to look for him in the sides of the image).
I can lie to you with a good probability – even if I haven’t found him my general statement may be right.
And finally, the biggest drawback is that you may still not be convinced (i.e. we have to wait for you to find him as well).

Suddenly it hits me! I know what to do! I take a big white piece of paper (much bigger than the original image, say twice as big in every direction). I then place the original image at a random location beneath it and cutout Waldo’s location from the paper. I then place another big piece of paper on top of everything (this time, without the cutout):

In my illustration the covering papers have the same size as the original image, but they should be much bigger in reality.

You then choose a number: 1 or 2. If you choose 1, I remove both pieces of paper, revealing to you the entire image. If you choose 2, I remove only the top piece of paper, revealing to you that I know where Waldo is, as such:

(click thumbnail for big image)

What does this accomplish? Well, first of all, if indeed I know where Waldo is I can answer both of your questions. If, on the other hand, I was lying, and I put the original image under the papers, then I could not have cut out Waldo’s location. In order to avoid this, I could have replaced the underlying image all together (as you do not see it at all!) with another image, in which I know where Waldo is. In this case, should you ask me to show you where Waldo is, I will be able to do so, but I will get busted if you selected 1, and made me reveal to you the entire image.

What we got is this:

If I know where Waldo is, I can deliver a correct answer that you can verify 100% of the time.
If I am lying and I do not know where Waldo is, you will bust me (no matter what I’ll do) with a probability of at least 1/2.

Well, being busted with a probability of only 1/2 is not very bad, but notice that we can repeat the process any number of times (each time placing the white papers at different locations over the original image). My chances of being correct are independent in each of the times, and so I will get busted with a probability (1/2)^n (where n is the number of iterations). If n is chosen high enough, you can be quite certain that I am not lying (10 iterations suffice for the probability of missing a liar to be less the 1/1000).

Note that the process is not really “Zero Knowledge” as by looking at the cutout version of Waldo you get an idea of his immediate surroundings and of his own posture (indeed, my cutout image above helps a little in finding him).

In the next installment of the series I will further discuss the notion of ZKP, especially in relation to solutions of NPC problems.

To be continued…

Understanding Soccer

yaniv — Sun, 03 Jun 2007 16:44:02 +0000

In 1998 (I can’t believe it was so long ago!) I was working at MATE (Media Access Technologies), an Israeli start-up at the time, specialising in face recognition and video searching. It was then that I came to know one of the coolest tricks in image recognition - the Hough Transform (pronounced “huf transform“).

In this article, I will demonstrate the algorithm with the following example:

Say you want to write a program that watches soccer. The input to your program is a sequence of frames from the game. The output should be the locations of all the players at all times. Given a frame of soccer, you first need to locate the players in the frame (i.e. you should know that a certain player occupies a certain set of pixels). Next, you need to know what location in the world is represented by those pixels. In order to do that, you need to figure out where the camera is looking at. In this article I will focus on this part of the problem (understanding the view point and direction). I will not deal with the (easier) part of determining what pixels are occupied by players.

Consider the following image of Ronaldinho:

I manually transformed it to this:

(the portion of the image covered by grass). I think it is clear how this step can be automatized.

Next I applied an automatic algorithm that got me this:

(the same image converted to black and white, for all the pixels that were bright enough). My algorithm simply turned all the pixels that are brighter than the color (80,80,20) () on a per channel basis to white, and all the other pixels to black. Note that my threshold color requires a high value of the red component, which is not present in the green grass, in order to turn the grass itself to black.

Lets return to our original problem – determining the point and direction of view (POV). We have big clues in the picture for determining the POV, namely the lines on the ground. Note that from the lines on the ground in the black and white picture, it is easy to see that Ronaldinho is standing in the center of the field. Can we somehow detect them?

Lets formulate the following abstact problem:

You are given a matrix M over Z2 (i.e. a matrix of bits). The matrix represents an image. Pixel Pij of the image is white if Mij is 0 and black if Mijis 1. The image contains straight lines. The lines are not perfect (i.e. they may overlap, they may not be continuous and there is a lot of noise). Your goal is to develop an efficient algorithm that receives the matrix and outputs the line equations.

Please take some time to think about this before continuing to read. If you spend enough time thinking about it, you will probably reach the conclusion that this is a really though problem.

A solution to this problem is the Hough Transform. The Hough Transform consists of noticing a duality of the Euclidean plane. We Consider a the straight line equation:

y = m * x + b

In this equation m and b are constants and x and y are variables. But what happens if we make x and y constants and m and b variables?

Well, the set of m’s and b’s that satisfy the equation represents all the lines that pass through our point (x, y)!

It is easy to see that for each point in the x-y plane we get a line in the b-m plane, namely, the line:

b = -x * m + y

Now each line in the x-y plane is matched with a point in the b-m plane! The duality depicted:

So the algorithm basically says:

Run through the original matrix.
For each pixel (x,y) that is white draw the line that matches it in the dual plane.
Then count the number of line intersections in the dual plane.
All points at which there is an intersection of more than T lines (these correspond to lines in the x-y plane on which we have more than T points in our original image) are considered valid lines.

To better illustrate the algorithm, I jotted down a few lines of python. The results follow (I must say that I was really surprised that it worked perfectly the first time around! All the threshold values I picked were somehow correct on the first go ).

Note that there is an implementation problem with using the b-m plane - Precision issues. For example, a near vertical line has huge values of m. In order to solve this, I used another duality (which is completely equivalent): the duality of the x-y plane and the a-c plane, where a is the angle of the line with the x axis (i.e. m = tan(a)), and c is b*cos(a).

The Hough map (i.e. the result of the Hough Transform) my script produced is:

Since it is kind of hard to distinguish the different color values, I generated a more human-friendly version (all the values are multiplied by 15):

And finally, the result (after converting back to the x-y plane):

And the same result overlayed on the original image:

A perfect match!

This is a big subject, and I only covered a small part of it. I will update this article in the near future with a more detailed analysis. If you have specific questions please post them as comments and I promise I will answer!

Meanwhile, let me just point out that the same method can be used to detect other shapes (e.g. circles). The generalization is trivial.

I will finish by showing the results of running the script on another image. Note that in this case, the threshold values are a little off. The results are nevertheless impressive:

The black and white image generated was:

The Hough map:

The result:

And, finally, the overlayed result:

To be updated…

Perl vs. Python One-Liner

yaniv — Sat, 19 May 2007 22:45:11 +0000

A few years ago a friend of mine asked me the following Perl riddle. Unfortunately, in order to solve it you must know Perl. As I like Python much better, I translated the riddle to Python. Attached are both versions.

I admit the Perl version is a bit more cryptic and if you know both Perl and Python you should try to solve the Perl version (but use Python for everything else in life ).

Oh, and try to solve the riddle without running it (run it only as a last resort).

perl -wle 'print "True" if (1 x shift) !~ /^1?$|^(11+?)\1+$/' [number]

python -c "import sys, re; print None == re.match('^1?$|^(11+?)\\1+$','1'*int(sys.argv[1]))" [number]

To alleviate all doubt – [number] denotes a numeric command line argument (e.g. 17).