Monday, July 04, 2016

Edit Distance

We use edit distance to do approximate matching between strings. It's time to get more familiar of it.

It is also known as Levenshtein distance, It is usually divided by maximum length of the two strings to get a percentage of similarity between two strings. getLenvenshtein(s1,s2)/max(s1.length, s2.length).


  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
package string;

import java.util.Arrays;
import java.util.Objects;

/**
1.If last characters of two strings are same, nothing much to do. 
 Ignore last characters and get count for remaining strings. 
So we recur for lengths m-1 and n-1.
2. Else (If last characters are not same), we consider all operations on 
‘str1′, consider all 
three operations on last character of first string, recursively compute 
minimum cost for all three operations and take minimum of three values.

Insert: Recur for m and n-1
Remove: Recur for m-1 and n
Replace: Recur for m-1 and n-1

https://en.wikipedia.org/wiki/Levenshtein_distance

 * @author Andrew Ma
 *
 */
public class EditDistance {
 static int min(int x, int y, int z){
  return Math.min(x,Math.min(y, z));
 }
 //this method is from right to left, it can be left to right too
 static int editDistanceRecursive(String s1, String s2, int s1Len, int s2Len){
  if(s1Len==0)return s2Len;
  if(s2Len==0)return s1Len;
  //last characters are same, do not increase the distance, moving to next characters and continue
  if(s1.charAt(s1Len-1)==s2.charAt(s2Len-1))
   return editDistanceRecursive(s1,s2,s1Len-1,s2Len-1);
  //for last characters are not same
  return 1+min(
    //remove the difference from s1, mark one difference, then continue to check rest of strings
    //after removing, it 's no guaranteed that new last is same as last one in second string
    //so this operation does not remove last character in s2
    editDistanceRecursive(s1,s2,s1Len-1,s2Len),
    //insert a matched character to s1, mark one difference, then continue to check rest of strings
    //after adding one character, s1's length becomes s1Len+1, we adding one distance for s1Len+1 and s2Len
    //then move to next positions that are s1Len and s2Len-1
    //(when we add, we add same character, so guarantee the last ones match)
     editDistanceRecursive(s1, s2, s1Len,s2Len-1),
     //replace, now we now last ones are same after adding one edit, then we can move to next positions
      editDistanceRecursive(s1,s2,s1Len-1,s2Len-1));
  
 }
 /**
  * straight forward way of dp to remove overlapping calculation of subproblems
  * @param s1
  * @param s2
  * @param s1Len
  * @param s2Len
  * @param dp
  * @return
  */
 static int editDistanceRecursiveDP(String s1, String s2, int s1Len, int s2Len,int[][] dp){
  if(s1Len==0)return s2Len;
  if(s2Len==0)return s1Len;
  if(dp[s1Len][s2Len]>0)return dp[s1Len][s2Len];
  //last characters are same, do not increase the distance, moving to next characters and continue
  if(s1.charAt(s1Len-1)==s2.charAt(s2Len-1)){
   dp[s1Len][s2Len] = editDistanceRecursive(s1,s2,s1Len-1,s2Len-1);
   return dp[s1Len][s2Len];
  }
  //for last characters are not same
  dp[s1Len][s2Len] = 1+min(
    //remove the difference from s1, mark one difference, then continue to check rest of strings
    //after removing, it 's no guaranteed that new last is same as last one in second string
    //so this operation does not remove last character in s2
    editDistanceRecursive(s1,s2,s1Len-1,s2Len),
    //insert a matched character to s1, mark one difference, then continue to check rest of strings
    //after adding one character, s1's length becomes s1Len+1, we adding one distance for s1Len+1 and s2Len
    //then move to next positions that are s1Len and s2Len-1
    //(when we add, we add same character, so guarantee the last ones match)
     editDistanceRecursive(s1, s2, s1Len,s2Len-1),
     //replace, now we now last ones are same after adding one edit, then we can move to next positions
      editDistanceRecursive(s1,s2,s1Len-1,s2Len-1));
  return dp[s1Len][s2Len];
  
 }
 /**
  * bottom up dp. advancing to here after understanding the recursive algorithm
  * @param s1
  * @param s2
  * @return
  */
 static int editDistanceDP(String s1, String s2){
  Objects.requireNonNull(s1);
  Objects.requireNonNull(s2);
//  if(s1.length()==0)return s2.length();
//  if(s2.length()==0)return s1.length();
  //i,j means difference so far up to i-1 in s1 and j-1 in s2
  // so the last examined positions are last characters in both string, in dp[][], that is dp[m][n]
  int[][] dp = new int[s1.length()+1][s2.length()+1];
  for (int i=0; i<=s1.length(); i++)
        {
            for (int j=0; j<=s2.length(); j++)
            {
             if(i==0)
              dp[i][j]=j;
             else if (j==0)
              dp[i][j]=i;
             //when it runs to here , i and j would be greater than 0
             //i-1 and j-1 are real index to strings
             //0 is just for dp[][]
             else if (s1.charAt(i-1)==s2.charAt(j-1))
              dp[i][j] = dp[i-1][j-1];
             else{
              //edit operation happens on s1
              dp[i][j] = 1+min(
                dp[i][j-1],//remove:means i skip one to next
                dp[i-1][j],//insert: means i keeps same, j goes to next one
                dp[i-1][j-1]//replace: both skip one
                );
             }
            }
        }
  return dp[s1.length()][s2.length()];
 }
 public static void main(String[] args) {
  String s1 = "andrewma";
  String s2 = "andyMa";
  int[][] dp = new int[s1.length()+1][s2.length()+1];
  for(int i=0;i<dp.length;i++)
   Arrays.fill(dp[i], -1);
  System.out.println(editDistanceRecursive(s1, s2, s1.length(), s2.length()));
  System.out.println(editDistanceRecursiveDP(s1, s2, s1.length(), s2.length(),dp));
  System.out.println(editDistanceDP(s1, s2)); 
 }

}
Below is from wiki, easiler to understand. But I have a different understan don deletion and insertion operation.
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
 static int editDistanceDPWiki(String s1, String s2){
  Objects.requireNonNull(s1);
  Objects.requireNonNull(s2);
   // for all i and j, dp[i,j] will hold the Levenshtein distance between
    // the first i characters of s and the first j characters of t
    // note that dp has (m+1)*(n+1) values
  int[][] dp = new int[s1.length()+1][s2.length()+1];//initially they are 0s
  //target is empty string. source can be transformed by deleting all characters
  for (int i=1; i<=s1.length(); i++)
   dp[i][0]=i;
  //source is empty. target can be transformed by dropping all characters
  for (int i=1; i<=s2.length(); i++)
   dp[0][i]=i;
  
  for (int j=1; j<=s2.length(); j++)
        {
            for (int i=1; i<=s1.length(); i++)
            {
             if (s1.charAt(i-1)==s2.charAt(j-1))//
              dp[i][j] = dp[i-1][j-1];//no difference
             else{
              //edit operation happens on s1
              dp[i][j] = 1+min(
                dp[i][j-1],//deletion:
                dp[i-1][j],//insertion: 
                dp[i-1][j-1]//replace: both skip one
                );
             }
            }
        }
  return dp[s1.length()][s2.length()];
 }
There is another way to use two one dimension arrays instead of a matrix to save space as demonstrated in the wiki page. Other places to find good information are from Algorithms on Strings, Trees and Sequences by Dan Gusfield and Chas Emerick's implementation of the Levenshtein distance algorithm from http://www.merriampark.com/ld.htm

Length of Longest Increasing Sequence

In term of figuring out length of LIS, here is a very impressive way to solve it. since it does binary search for each of element, the time complexity is O(nlgn).

The idea is to keep a working array sorted and it's working length is controlled by if a bigger number is introduced into the array.

Binary search utility is supported by Java's Arrays.binarySearch function, which suggest an index the new element should be put to. In the algorithm, for same position, number will be overwritten by smaller numbers, so the content is the the increasing sequence we are looking for, but the length of it is correct. Smart!

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
package array;

import java.util.Arrays;

public class LongestIncreasingSubsequence {
public static void main(String[] args){
 int a[] = new int[]{7,6,2,4,3,8,9,1,15,14,13,5};
 System.out.println(lis(a));
}
/**
 * this algorithm is to keep updating a sorted array, in the sorted array,
 * the length reflects the max length of increasing sequence, but the content
 * can be overwrite over the time, so that the content is not the sequence that should be printed.
 * @param a
 * @return
 */
static int lis(int a[]){
 if (a==null || a.length==0)return 0;
 int dp[] = new int[a.length];
 int len=0;
 for(int x: a){
  //return position it should be at, limit the len to known increasing length
  int i = Arrays.binarySearch(dp, 0, len, x);
  i=(i<0?-(i+1):i);//it'd be negative all any new element
  //this is where the overwriting happens, 
  //it will overwrite the same index with smaller value
  //all the numbers in the sequence can be overwritten by smaller numbers, 
  //but it since it's overwritten by smaller numbers, it won't change the sorting order of the one without being overwritten
  //only the growth of len will change the length of increasing length of sub sequence
  dp[i] = x;
  if(i==len)len++;
 }
 return len;
}

}

Saturday, July 02, 2016

The Maximum Subarray

The Maximum Subarray


Given an array  of  elements, find the maximum possible sum of a
  1. Contiguous subarray
  2. Non-contiguous (not necessarily contiguous) subarray.
Empty subarrays/subsequences should not be considered.
Input Format
First line of the input has an integer .  cases follow.
Each test case begins with an integer . In the next line,  integers follow representing the elements of array .
Constraints:
  • 1<=T<=10
  • 1<=N<=100000
  • -10000<=ai<=10000
The subarray and subsequences you consider should have at least one element.
Output Format
Two, space separated, integers denoting the maximum contiguous and non-contiguous subarray. At least one integer should be selected and put into the subarrays (this may be required in cases where all elements are negative).
Sample Input
2 
4 
1 2 3 4
6
2 -1 2 3 4 -5
Sample Output
10 10
10 11
Explanation
In the first case:
The max sum for both contiguous and non-contiguous elements is the sum of ALL the elements (as they are all positive).
In the second case:
[2 -1 2 3 4] --> This forms the contiguous sub-array with the maximum sum.
For the max sum of a not-necessarily-contiguous group of elements, simply add all the positive elements.

This is not at all easy, it needs to calculate maximum values for both consecutive and sparse maximum summaries, and you need to know that famous solution to max subarray. Also finally you need to pay attention to how sparse maximum summaries can be calculated.
package DP;

import java.util.Scanner;

/**
 * Created by AndrewLocal on 7/2/2016.
 */

public class MaxSubarray {
    public static void main(String[] args) {
        /* Enter your code here. Read input from STDIN. Print output to STDOUT. Your class should be named Solution. */
        Scanner in  =  new Scanner(System.in);
        int N = in.nextInt();in.nextLine();
        for(int i=0;i<N;i++){
            int n = in.nextInt();in.nextLine();
            int a[] = new int[n];
            for(int j=0;j<n;j++)
                a[j]=in.nextInt();
            if(in.hasNextLine())in.nextLine();
            process(a);
        }
        in.close();
    }
    private static void process(int[] a){
        int cSum = a[0];
        int sSum[] = new int[a.length];//sparse, adding numbers only when it makes total bigger
        int cSumMax = a[0];
        //cSum[0]=a[0];
        sSum[0]=a[0];
        for(int i=1;i<a.length;i++){
//-1 2: -1+2>-1 but I'd rather not add -1 to 2 because 1 is less than 2
            sSum[i]=Math.max(sSum[i-1],Math.max(sSum[i-1]+a[i],a[i]));//sparse sum, increase or keep the same
            cSum=Math.max(cSum+a[i],a[i]);//max sum ending here
            cSumMax = Math.max(cSumMax,cSum);//max so far is cSumMax
        }

        System.out.format("%d %d\n",cSumMax,sSum[a.length-1]);
    }

}

Friday, July 01, 2016

String Practice

Generate all sub-sequences of length K


From http://introcs.cs.princeton.edu/java/23recursion/Subsequence.java.html



 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
public class Subsequence { 

    public static void print(String prefix, String remaining, int k) {
     if(k>prefix.length()+remaining.length())return;
        if (k == 0) {
            System.out.println(prefix);
            return;
        }
        if (remaining.length() == 0) return; 
//pick the first character in remaining, combine with remaining of remaining
//this is to sequentially build up the subsequence until the k length 
        print(prefix + remaining.charAt(0), remaining.substring(1), k-1); 
//then another possible is to skip one character in remaining, and combine with rest of remaining
        print(prefix, remaining.substring(1), k);
    }


    public static void main(String[] args) { 
        String s = "abcd";
        int k =3;
        print("", s, k);
    }

}
Longest common subsequence This is a basic DP problem. code can be very concise after understanding it. https://www.youtube.com/watch?v=NnD96abizww http://www.geeksforgeeks.org/dynamic-programming-set-4-longest-common-subsequence/ The following code example is form http://introcs.cs.princeton.edu/java/96optimization/LCS.java.html
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
package string;

public class LCS {

     public static String lcs(String s1,String s2) {
         int M = s1.length();
         int N = s2.length();

         // opt[i][j] = length of LCS of x[i..M] and y[j..N]
         int[][] opt = new int[M+1][N+1];

         // compute length of LCS and all subproblems via dynamic programming
         //this is the basic formula:
         //this is also working from back to start. usually it is from start to end
         for (int i = M-1; i >= 0; i--) {
             for (int j = N-1; j >= 0; j--) {
                 if (s1.charAt(i) == s2.charAt(j))
                     opt[i][j] = opt[i+1][j+1] + 1;
                 else 
                     opt[i][j] = Math.max(opt[i+1][j], opt[i][j+1]);
             }
         }
         //version from start to end
         /*
          for (int i = 1; i <M; i++) {
             for (int j = 1; j <N; j++) {
                 if (s1.charAt(i) == s2.charAt(j))
                     opt[i][j] = opt[i-1][j-1] + 1;
                 else 
                     opt[i][j] = Math.max(opt[i-1][j], opt[i][j-1]);
             }
         }
          */
         // recover LCS itself and print it to standard output
         StringBuilder sb = new StringBuilder();
         int i = 0, j = 0;
         while(i < M && j < N) {
             if (s1.charAt(i) == s2.charAt(j)) {
                 sb.append(s1.charAt(i));
                 i++;
                 j++;
             }
             else if (opt[i+1][j] >= opt[i][j+1]) i++;//means it is from i++
             else                                 j++;//means it is from j++
         }
         return sb.toString();
     }
}
Longest Common SubString similar but easier than common subsequence, still use 2D matrix, compare if m[i,j]=(s1.charAt(i)==s2.charAt(j)) and go diagonally until it is not true. repeat that process to find longest one.