I am confused with GATK Multi-threading options, -nt / --num_threads controls the number of data threads sent to the processor, -nct / --num_cpu_threads_per_data_thread controls the number of CPU threads allocated to each data thread.

What’s data threads?

Here is an answer from Geraldine Van der Auwera, PhD

In the meantime, what you need to know is that -nct is the number of CPU threads, ie threads that can be run by different cores if you have a multicore CPU, while -nt is the number of data threads, ie number of “clones” of the GATK that are run in parallel on your machine.

So nt is based on how many copies you want to run in the same time.

nct can control the CPU threads. Take CB60-G16 Network Server for example,

Each node has two CPUs (10 cores, 20 threads).

If I use -nct 10, it means I use 10 threads on one node. -nt 10 means I use 10 threads for one data thread(one sample). nt might cause an increasing in memory usage, each data thread would be allowed with the same memory size.

HotNet2 is a Python based software to identify subnetworks of gene interaction network with mutation infomation.

I read the Algorithm last month and got a basic knowledge of random walk.

Random walk with restart

concept

For gene(protein) interaction network, random walk starts from a protein g and at each time step moves to one of the neighbors with the probability \(1-\beta\) (\(0 \leq \beta \leq 1\)).

The walk can also restarts from g with probability \(\beta\). This process is defined by a transition matrix \(W\).

\(deg(j)\) is the number of neighbors (the degree) of protein \(g_{i}\) in the interaction network.

\(\beta\) represent the probability with which the walk starting at \(g_{i}\) is forced to restart from \(g_{i}\).

The random walk will reaches a stationary discribution described by the vector \(\vec{s}_{i}\)

When \(\vec{s}^{\prime}_{i}=\vec{s}_{i}\), we can get

where\(\vec{e}_{i}\) is the vector with a 1 in position \(i\) and 0 is in the remaining positions.

This part \(\beta(I-(1-\beta)W)^{-1}\) is called diffusion matrix \(F\).

Note that \(\vec{s}_{i}\) is the \(i^{th}\) column of \(F\).

parameter

To calculate the diffusion matrix, we need know the value of \(\beta\). In HotNet2, they chose \(\beta\) to balance the amount of heat that diffuses from a protein to its immediate neighbors and to the rest of the network.

There is another parameter \(\delta\), it is the edge wight parameter. It’s used to make sure the HotNet2 will not find large subnetworks using random data.

For more detail of these parameters, please see the supplementary of this paper.

sample size problem

I ask a question about sample size on HotNet google group.

I read the HotNet2 paper and find the size of samples used in this paper is very large. If my sample size is small ( for example, 10 samples or 20 samples ), could I use hotnet2 to do the pathway analysis? Is there any baseline of sample size ?

As I understand, you can use hotnet2 with whatever sample size you have. For example if you take a look on this analysis here: https://cs.brown.edu/research/pubs/theses/ugrad/2014/jain.pdf you can see that they used p-values as input, thus hotnet itself does not depend on the sample size. This means, that you have to take into account sample size caused biases at the p-value calculation. Therefore, if your sample size is low, you might want to consider more robust ways to calculate p-values, e.g. some rank based approaches (for example Rank Product).

Answer by Akos Tenyi

John Nash and his wife were killed in crash.

For me, it marks the end of an era.

Forever is an American drama television series about Dr. Henry Morgan, an immortal medical examiner in New York City. On May 7, 2015, ABC cancelled Forever after one season.

My note

I like Dr. Morgan’s overlapping sound in each episode and copied those to here.

Enjoy!

s1e01

I have seen a lot of death, a lot of pain, a lot of suffering. But I’ve also seen a lot of life… A lot of beauty, a lot of wonder. It’s not the number of years we live that matters. Our lives just add up to a series of moments. we never know when or where they’ll happen. But they stick with us, marking our souls forever. The problem with living for 200 years isn’t the loneliness or the pain or the loss. Okey, sure, it is those things. But what really gets you is when life ceases to surprise you. I’ve spent my entire life studying the human body, and I can say with scientific certainty that what keeps us alive, more important than blood or oxygen or even love… is hope.

s1e02

Noting can erase the pain of losing someone you love. You carry it with you for the rest of your life. However long that might be. The best you can hope for is that over time, the wound begins to heal. But no matter how strong we are, no matter how hard we fight… the scar always stays with us.

s1e03

As sad and dreadful as death may be, it forces us to cherish every moment because the truth is… life is precious because it’s finite.

When you are immortal, you have to be reminded of beauty. Days stretch into years, stretch into centuries. Time can lose its meaning.

So however long you live, even though that might prove a very, very long time… you must tell yourself to live every day…every hour, as if it could be your last.

s1e04

For the dead, all is forgotten. But all is not lost, because they’re remembered by us. Some we think of fondly, others less so. A few are remembered by many, and some by only a few.

s1e05

Every generation likes to think They’re improving upon the last, that progress is inevitable, That the future holds less misery and suffering than the past. But the truth is, some things never change. History has a way of repeating itself. It’s just most people don’t live long enough to see it happen.

But some things do change. Years ago, boxers fought bare-knuckled. In 1855, one bout lasted 6 hours,15 minutes. Back then, the so-called ring was an actual ring,made up of rowdy, bloodthirsty spectators. That’s not to say that fights are fair today. The big guy’s gloves are still loaded a little heavier, but it’s no reason to stop fighting.

s1e06

Murder is never an easy thing to look at. When you examine the body of a victim,you don’t see a reflection of their life, but the person who took it. You see whether the killer felt anger or betrayal, or worse… if they felt nothing at all.

We may never understand those who kill so easily for spectacle. But perhaps that’s why we’re so fascinated with them. We read about their exploits out of curiosity. Our fascination with the macabre makes it all seem unreal.But the truth is that everyone has the capacity to kill. The real question is, how do you live with yourself afterward?

s1e07

Every crime, like every person, has its secrets. Some are never revealed, while others are hidden in plain sight.

We can train our bodies to forget. But not our minds. And one thing living forever teaches you is that your past, your secrets, mark you always, like it or not.

There are things for all of us that even all the time in the world wouldn’t be able to erase.

s1e08

Sex, drugs, skydiving.It’s strange that the things we do for a thrill, in order to make ourselves feel alive, are the same things that may kill us. The same is true when it comes to human relationships. The people we love the most are the ones capable of hurting us the most deeply.

Our body feels pain to warn us of danger. But is also reminds us we’re alive, that we can still feel. That’s why some of us seek it out while others choose to numb it. Solitude has always been my analgesic of choice. But what if feeling nothing is the worst pain of all? What if the sharing of pain connects us to others and remainds us that none of us is alone, as long as we can feel?

s1e09

No matter how we live or die, we all end the same, in silence. All of our hopes and dreams in life become mere echoes of a tale cur short. But if we’re lucky enough, our stories live on. Our song finds voice in the hearts of those who remember us and loved us.

It has been said that a man is not dead while his name is still spoken, that we are only truly gone when we’ve disappeared from the memories of those who loved us, meaning a great artist never dies. As long as his books are read, his paintings admired, as long as our songs are sung, we may each of us live forever. As for me, maybe it’s time to learn a new song. Fortunately, I have an excellent teacher.

s1e10

What we try to hide about ourselves in life is revealed in death…our fears, our insecuritires, but most of all, our secrets.

The best impostors have many gifts. They can change their voice, their look, their manner. But there’s a rule every impostor learns. The one truth we cannot hide is love.

s1e11

We all have secrets. There is nothing wrong with that, but we all need a confidante, a friend to share our secretes with. In a way, the shared secret tells us who our real friends are. They are the people we entrust the most.

s1e12

There is risk anytime we venture into the unknown, whenever we are compelled, for whatever reason to push away the safety of the familiar of family and home. And there is the notion that this impulse is, indeed, for the best, that whatever doesn’t kill us makes us stronger. Unless, of course, it does.

Committing fully to the protection of another can often engender a kind of paradox. In doing so, we are tempted to put ourselves at risk the very thing most likely to cause pain in those whom we are trying so desperately to keep pain from. Over time, however, one learns that the choices of those we love are impossible to control.

s1e13

It is often when one least expects it that fate arrives at our door. We can’t foresee it any more than we can escape it. No matter how hard we run or hide, fate, like death, will always find us.

Fate can be cruel. A minute here, a minute there, and lives can be lost. But fate also gifts us moments of immpossible grace. Moments in time when we come together and connect in a way that is profound, chance occurrences so precious that they almost make one believe in the divine, that there is someone out there looking after us.

s1e14

The average person is said to apologize 12 times a day. Often, it’s to atone for a past wrong, an attempt to heal an old wound. But there are some wounds that can never be healed. They run too deep.

abe: Appreciate the present, focus on cheerier things.

henry: Sometimes, the people we love are the ones who anger us the most.

henry’s father: There are two things we are charged with in this life, living with our mistakes and endeavoring to learn from them, to be better men because of them.

Atonement can take many forms, a heartfelt apology, a grand gesture, a silent prayer, or something more complex, more gray, more difficult to decipher. And while it’s true that no amount of atonement can ever change the past, it does have the power to heal us, to restore relationships and connections we thought were lost forever.

s1e15

Why do we feel compelled to trace our roots? Those names in the family tree are just that…names…and yet the kinship we feel to them is powerful. Does the fact that we’re rooted in history make us feel more secure in our present? But what if our family trees are more like vines, twining aimlessly throughout the ages? The truth is, each of us is related. It’s just a question of how far back you trace your family tree. Deep down, all of us have shared blood in our veins. Although, individual tastes may vary. And if we’re all related, then all of us have royal blood, which is why every child should be treated as a king or queen. No matter how old our children may be.a

s1e16

we are told that those who ignore the past are doomed to repeat it, while those who dwell in the past would love nothing more than to repeat it over and over again. But memories, like photographs, have a way of fading with time. Some memories are so powerful that they never fade. Memories which become more powerful, every time we remember them. Thus making our present lives seem a little more faded. But let’s not think about that now.

s1e17

Perhaps privacy is a thing of the past. In the future, everything about us will be available with the touch of a button. Who’s fallen in love, who’s gotten sick, who’s having an affair. Maybe it’s a good thing – A future without secrets. Of course, those of us with a few more years value our privacy. We’ve made our mistakes and survived them. It’s not that we want to keep these secrets from others. Usually, we’re keeping them from ourselves.

s1e18

Every person who has lived long enough has something in their past that must be forgotten, a moment when they’ve failed, or when they simply went right instead of left, and that simple choice changed their life for the worse, so we cover it up and forget. But deep down, we know it will come back to haunt us.

A great alienist once told me, that we must face those moments we choose to forget. Only by confronting our fears are we able to grow, to change, and to be able to face our next.

s1e19

They say memories make us who we are, that the past defines us, but we can’t forget to grow, evolve, because sometimes a memory can be so powerful that we get stuck in it frozen in a moment.

Yes, some memories are precious and we need to hang on to them. But Emily Dickinson wrote,”Forever is composed of nows”. And she’s right. If we root ourselves too deeply in the past, we’ll miss what’s right in front of us.

s1e20

Every human being on this earth knows that life is not without pain. For most of us, it’s an inconvenience, a trial to be endured or avoided. But there are a select few who seek it out, who use it as a tool to create something so beautiful that it transcends this world and makes them immortal.

There are moments in all of our lives when we are compelled to ask ourselves, how far are we willing to go for what we want? How much are we willing to risk, to sacrifice, to endure? Because it’s one thing to yearn for something…but quite another to find the strength to achieve it.

s1e22

None of us can make it through this life without suffering some kind of pain. Having lived through my fair share, I can tell you the most difficult to endure is loneliness. Adam was right. Life is a game and one that we must play. Now matter how careful we are, there is simply no way to go through this life unscathed. But, fortunately for us, it’s a game we don’t have to play alone.

Definitely, animation package do not provide any interface for shiny app. Take grad.desc() for example, if we want to build an dynamic app that control all arguments of grad.desc() on app panel, we need split the function into sections and rewrite each section into server.R file.

I have another simple method to generate anmiation with shiny app. The method is generate pictures and use shiny app to show all pictures as slides.

Firstly, I need generate animation frames, use saveHTML() can get all pictures in images folder.

saveHTML({
         ani.options(interval = 0.3)
         grad.desc()
     }, img.name = "grad.desc", htmlfile = "grad.desc.html", ani.height = 500, 
         ani.width = 500, title = "Demonstration of the Gradient Descent Algorithm", 
         description = "The arrows will take you to the optimum step by step.")

Then, write a slide-app.

ui.R

library(shiny)
shinyUI(fluidPage(
  # Application title
  titlePanel("Grad.desc demo"),
  # Sidebar with a slider input for the number of frames
  sidebarLayout(
    sidebarPanel(
      sliderInput('myslider', 
                  'Steps', 
                  min=1, 
                  max=35, # count all frames(pictures) 
                  value=1, 
                  animate=animationOptions(interval = 0,loop=T)
      )
    ),  
    # Show ui
    mainPanel(
      uiOutput("ui")
    )
  )
))

server.R

library(shiny)
shinyServer(function(input, output, session) {
  imgurl <- reactive({
    i=input$myslider
    return(paste0("./images/grad.desc",i,".png")) #the path of pictures
  })
  output$ui <- renderUI({
    tags$div(
      tags$img(src = imgurl())
    )
  })
})

The directory structure.

.
|-- server.R
|-- ui.R
`-- www
    `-- images # images folder

Finally, runApp("demo") you would see the demo app running successfully!