Page Comparison

...

SVD on the entire 27k by 511 patients M value matrix, plot 1st eigenarray (u matrix) and "color" the points according to the dye with with each CpG was labeled:

Image Added

M value: analysis of red probes, identification of adjustment variables

...

Removing the batch effect didn't completely remove the center and the row/column effect. Day, month and year of shipment have been taken care of. Next remove the batch the the plate row effect. We identified with Justin the the entire 0652 batch doesn't have any plate row or column information:

> table(batch, adj$plate_row)

batch  A B C D E F G H
  0359 5 5 5 5 5 5 4 4
  0402 5 4 4 5 5 5 4 4
  0432 6 5 5 5 6 5 6 5
  0460 6 6 6 6 6 6 6 5
  0475 6 6 6 6 5 6 6 5
  0501 3 3 3 3 3 3 2 2
  0536 6 6 6 6 6 6 6 5
  0563 6 6 6 6 6 6 6 5
  0581 6 6 6 6 6 6 6 5
  0652 0 0 0 0 0 0 0 0
  0667 6 6 6 6 6 6 5 5
  0708 6 6 6 6 5 6 5 5
  0807 1 0 0 2 0 1 0 0
> table(batch, adj$plate_column)
batch  1 2 3 4 5 6
  0359 8 8 8 8 6 0
  0402 8 8 6 8 6 0
  0432 5 8 7 8 8 7
  0460 8 8 8 8 8 7
  0475 8 8 7 8 8 7
  0501 8 8 6 0 0 0
  0536 8 8 8 8 8 7
  0563 8 8 8 8 8 7
  0581 8 8 8 8 8 7
  0652 0 0 0 0 0 0
  0667 8 7 8 8 8 7
  0708 8 7 7 8 8 7
  0807 2 1 1 0 0 0
 

...

> mask<-batch!="0652"
> length(mask)
[1] 511
> table(mask)
mask
FALSE  TRUE
   43   468
> X<-model.matrix(~factor(batch[mask]) + adj$plate_row[mask])
> Xbcrw<-solve(t(X) %*% X) %*% t(X) %*% t(redBred[,mask])
> redBR<- redBred[,mask] - t(X %*% Xbcrw)

Percent variance explained after removing the batch and the plate row effects:

The first principal component is smaller but not significantly so. Lets look again at the variables:

Now remove the batch, plate row and plate column, look at the percent variance explained:

Image Added

Now the first principal component explains a little less than 15% of the overall variability. Correlation with the adjustment and bio variables, see if the center effect still present:

Finally, the center effect needs to go. Variables to adjust for: batch, center, plate row, plate column. Percent Variance explained after the adjustment:
Image Added Image Added

Under 14%! Still large effect, look at the variables:

Now lets take a look at the outliers of the first eigengene (patients 6 vs 367):

Image Added

I guess it doesn't look too terrible. I also tried to remove all the listed variables as well as the first principal component and here is what I got in terms of the percent variance explained and the outliers: Image Added  Image Added
To me it looks worse than with the first principal component. Final: remove the batch, center, plate row and plate column from the data. 

M value: analysis of green probes, identification of adjustment variables

Percent Variance explained:

Image Added

Remove the batch, center, plate row, plate column (also mask that one batch), looks at the first eigengene and the eigenarray:

Image Added Image Added

P values after the adjustment:

Look at the outliers:
Image Added
Weird!
Do one more test and remove the first eigengene together with the variables above:
Image Added Image Added

Conclusion

Now scale (center=TRUE, scale=TRUE) bot datasets (red and green probes), combine them together for network construction.